added {place.country_code} to template system

CHANGELOG.md

@@ -4,6 +4,22 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.23.3](https://github.com/RhetTbull/osxphotos/compare/v0.23.1...v0.23.3)
+
+> 22 March 2020
+
+- Initial version of templating system for CLI [`2feb099`](https://github.com/RhetTbull/osxphotos/commit/2feb0999b3f9ffd9a24e37238f780239a027aa49)
+- Added __str__ to place [`ad58b03`](https://github.com/RhetTbull/osxphotos/commit/ad58b03f2d31daf33849b141570dd0fb5e0a262e)
+- Test library updates [`e90d9c6`](https://github.com/RhetTbull/osxphotos/commit/e90d9c6e11fce7a4e4aa348dcc5f57420c0b6c44)
+
+#### [v0.23.1](https://github.com/RhetTbull/osxphotos/compare/v0.23.0...v0.23.1)
+
+> 21 March 2020
+
+- Fixed requirements.txt for bplist2 [`cda5f44`](https://github.com/RhetTbull/osxphotos/commit/cda5f446933ea2272409d1f153e2a7811626ada6)
+- Updated CHANGELOG.md [`b8da976`](https://github.com/RhetTbull/osxphotos/commit/b8da9765b8949eb90852d249c2877eeb1806d987)
+- Updated requirements.txt [`9da7ad6`](https://github.com/RhetTbull/osxphotos/commit/9da7ad6dcc021fdafe358d74e1c52f69dc49ade8)
+
 #### [v0.23.0](https://github.com/RhetTbull/osxphotos/compare/v0.22.23...v0.23.0)
 
 > 21 March 2020

README.md

@@ -14,8 +14,9 @@
     + [PhotosDB](#photosdb)
     + [PhotoInfo](#photoinfo)
     + [PlaceInfo](#placeinfo)
+    + [Template Functions](#template-functions)
     + [Utility Functions](#utility-functions)
-    + [Examples](#examples)
+  * [Examples](#examples)
   * [Related Projects](#related-projects)
   * [Contributing](#contributing)
   * [Implementation Notes](#implementation-notes)
@@ -214,19 +215,20 @@ Options:
                                   exiftool may be installed from
                                   https://exiftool.org/
   --directory DIRECTORY           Optional template for specifying name of
-                                  output directory.  See below for additional
-                                  details on templating system
+                                  output directory in the form
+                                  '{name,DEFAULT}'. See below for additional
+                                  details on templating system.
   -h, --help                      Show this message and exit.
 
 **Templating System**
 
 With the --directory option, you may specify a template for the export
-directory.  This directory will be appended to the export path specified  in
+directory.  This directory will be appended to the export path specified in
 the export DEST argument to export.  For example, if template is
 '{created.year}/{created.month}', and export desitnation DEST is
-'/Users/maria/Pictures/export',  the actual export directory for a photo would
-be '/Users/maria/Pictures/export/2020/March'  if the photo was created in
-March 2020.
+'/Users/maria/Pictures/export', the actual export directory for a photo would
+be '/Users/maria/Pictures/export/2020/March' if the photo was created in March
+2020.
 
 In the template, valid template substitutions will be replaced by the
 corresponding value from the table below.  Invalid substitutions will result
@@ -239,55 +241,77 @@ rendered name, escape the curly braces with \, for example, using
 '{created.year}/\{name\}' for --directory would result in output of
 2020/{name}/photoname.jpg
 
-In the current implementation, substitutions which have no value will be
-replaced by '_', for example, your template looked like
-'{created.year}/{place.address}' but there was no address associated with the
-photo, the resulting output would be: '2020/_/photoname.jpg'
+You may specify an optional default value to use if the substitution does not
+contain a value (e.g. the value is null) by specifying the default value after
+a ',' in the template string: for example, if template is
+'{created.year}/{place.address,'NO_ADDRESS'}' but there was no address
+associated with the photo, the resulting output would be:
+'2020/NO_ADDRESS/photoname.jpg'. If specified, the default value may not
+contain a brace symbol ('{' or '}').
 
-I plan to add the option to specify the value to be used for missing
-subsitutions in a future version. I also plan to extend the templating system
-to the exported filename so you can specify the filename using a template.
+If you do not specify a default value and the template substitution has no
+value, '_' (underscore) will be used as the default value. For example, in the
+above example, this would result in '2020/_/photoname.jpg' if address was null
+I plan to eventually extend the templating system to the exported filename so
+you can specify the filename using a template.
 
-Substitution          Description
-{name}                Filename of the photo
-{original_name}       Photo's original filename when imported to Photos
-{title}               Title of the photo
-{descr}               Description of the photo
-{created.date}        Photo's creation date in ISO format, e.g. '2020-03-22'
-{created.year}        4-digit year of file creation time
-{created.yy}          2-digit year of file creation time
-{created.mm}          2-digit month of the file creation time (zero padded)
-{created.month}       Month name in user's locale of the file creation time
-{created.mon}         Month abbreviation in the user's locale of the file
-                      creation time
-{created.doy}         3-digit day of year (e.g Julian day) of file creation
-                      time, starting from 1 (zero padded)
-{modified.date}       Photo's modification date in ISO format, e.g.
-                      '2020-03-22'
-{modified.year}       4-digit year of file modification time
-{modified.yy}         2-digit year of file modification time
-{modified.mm}         2-digit month of the file modification time (zero
-                      padded)
-{modified.month}      Month name in user's locale of the file modification
-                      time
-{modified.mon}        Month abbreviation in the user's locale of the file
-                      modification time
-{modified.doy}        3-digit day of year (e.g Julian day) of file
-                      modification time, starting from 1 (zero padded)
-{place.name}          Place name from the photo's reverse geolocation data
-{place.names}         list of place names from the photo's reverse
-                      geolocation data, joined with '_', for example, '18th
-                      St NW_Washington_DC_United States'
-{place.address}       Postal address from the photo's reverse geolocation
-                      data, e.g. '2007 18th St NW, Washington, DC 20009,
-                      United States'
-{place.street}        Street part of the postal address, e.g. '2007 18th St
-                      NW'
-{place.city}          City part of the postal address, e.g. 'Washington'
-{place.state}         State part of the postal address, e.g. 'DC'
-{place.postal_code}   Postal code part of the postal address, e.g. '20009'
-{place.country}       Country name of the postal code, e.g. 'United States'
-{place.country_code}  ISO country code of the postal address, e.g. 'US'
+Substitution                    Description
+{name}                          Filename of the photo
+{original_name}                 Photo's original filename when imported to
+                                Photos
+{title}                         Title of the photo
+{descr}                         Description of the photo
+{created.date}                  Photo's creation date in ISO format, e.g.
+                                '2020-03-22'
+{created.year}                  4-digit year of file creation time
+{created.yy}                    2-digit year of file creation time
+{created.mm}                    2-digit month of the file creation time
+                                (zero padded)
+{created.month}                 Month name in user's locale of the file
+                                creation time
+{created.mon}                   Month abbreviation in the user's locale of
+                                the file creation time
+{created.doy}                   3-digit day of year (e.g Julian day) of file
+                                creation time, starting from 1 (zero padded)
+{modified.date}                 Photo's modification date in ISO format,
+                                e.g. '2020-03-22'
+{modified.year}                 4-digit year of file modification time
+{modified.yy}                   2-digit year of file modification time
+{modified.mm}                   2-digit month of the file modification time
+                                (zero padded)
+{modified.month}                Month name in user's locale of the file
+                                modification time
+{modified.mon}                  Month abbreviation in the user's locale of
+                                the file modification time
+{modified.doy}                  3-digit day of year (e.g Julian day) of file
+                                modification time, starting from 1 (zero
+                                padded)
+{place.name}                    Place name from the photo's reverse
+                                geolocation data, as displayed in Photos
+{place.name.country}            Country name from the photo's reverse
+                                geolocation data
+{place.name.state_province}     State or province name from the photo's
+                                reverse geolocation data
+{place.name.city}               City or locality name from the photo's
+                                reverse geolocation data
+{place.name.area_of_interest}   Area of interest name (e.g. landmark or
+                                public place) from the photo's reverse
+                                geolocation data
+{place.address}                 Postal address from the photo's reverse
+                                geolocation data, e.g. '2007 18th St NW,
+                                Washington, DC 20009, United States'
+{place.address.street}          Street part of the postal address, e.g.
+                                '2007 18th St NW'
+{place.address.city}            City part of the postal address, e.g.
+                                'Washington'
+{place.address.state_province}  State/province part of the postal address,
+                                e.g. 'DC'
+{place.address.postal_code}     Postal code part of the postal address, e.g.
+                                '20009'
+{place.address.country}         Country name of the postal address, e.g.
+                                'United States'
+{place.address.country_code}    ISO country code of the postal address, e.g.
+                                'US'
 ```
 
 Example: export all photos to ~/Desktop/export, including edited versions and live photo movies, group in folders by date created
@@ -310,6 +334,7 @@ Example: export photos to file structure based on 4-digit year and full name of
 ## Example uses of the package 
 
 ```python
+""" Simple usage of the package """
 import os.path
 
 import osxphotos
@@ -350,34 +375,81 @@ if __name__ == "__main__":
 ```
 
 ```python
-""" Export all photos to ~/Desktop/export
-    If file has been edited, export the edited version, 
-    otherwise, export the original version """
+""" Export all photos to specified directory using album names as folders
+    If file has been edited, also export the edited version, 
+    otherwise, export the original version 
+    This will result in duplicate photos if photo is in more than album """
 
 import os.path
+import pathlib
+import sys
+
+import click
+from pathvalidate import is_valid_filepath, sanitize_filepath
+
 import osxphotos
 
 
-def main():
-    db = os.path.expanduser("~/Pictures/Photos Library.photoslibrary")
-    photosdb = osxphotos.PhotosDB(db)
-    photos = photosdb.photos()
+@click.command()
+@click.argument("export_path", type=click.Path(exists=True))
+@click.option(
+    "--default-album",
+    help="Default folder for photos with no album. Defaults to 'unfiled'",
+    default="unfiled",
+)
+@click.option(
+    "--library-path",
+    help="Path to Photos library, default to last used library",
+    default=None,
+)
+def export(export_path, default_album, library_path):
+    export_path = os.path.expanduser(export_path)
+    library_path = os.path.expanduser(library_path) if library_path else None
 
-    export_path = os.path.expanduser("~/Desktop/export")
+    if library_path is not None:
+        photosdb = osxphotos.PhotosDB(library_path)
+    else:
+        photosdb = osxphotos.PhotosDB()
+
+    photos = photosdb.photos()
 
     for p in photos:
         if not p.ismissing:
-            if p.hasadjustments:
-                exported = p.export(export_path, edited=True)
-            else:
-                exported = p.export(export_path)
-            print(f"Exported {p.filename} to {exported}")
+            albums = p.albums
+            if not albums:
+                albums = [default_album]
+            for album in albums:
+                click.echo(f"exporting {p.filename} in album {album}")
+
+                # make sure no invalid characters in destination path (could be in album name)
+                album_name = sanitize_filepath(album, platform="auto")
+
+                # create destination folder, if necessary, based on album name
+                dest_dir = os.path.join(export_path, album_name)
+
+                # verify path is a valid path
+                if not is_valid_filepath(dest_dir, platform="auto"):
+                    sys.exit(f"Invalid filepath {dest_dir}")
+
+                # create destination dir if needed
+                if not os.path.isdir(dest_dir):
+                    os.makedirs(dest_dir)
+
+                # export the photo
+                if p.hasadjustments:
+                    # export edited version
+                    exported = p.export(dest_dir, edited=True)
+                    edited_name = pathlib.Path(p.path_edited).name
+                    click.echo(f"Exported {edited_name} to {exported}")
+                # export unedited version
+                exported = p.export(dest_dir)
+                click.echo(f"Exported {p.filename} to {exported}")
         else:
-            print(f"Skipping missing photo: {p.filename}")
+            click.echo(f"Skipping missing photo: {p.filename}")
 
 
 if __name__ == "__main__":
-    main()
+    export()  # pylint: disable=no-value-for-parameter
 ```
 
 ## Package Interface
@@ -840,24 +912,38 @@ If overwrite=False and increment=False, export will fail if destination file alr
 Returns `True` if photo place is user's home address, otherwise `False`.
 
 #### `name`
-Returns the name of the local place as str.  This may be a street address (e.g. "2038 18th St NW") or a public place (e.g. "St James\'s Park").
+Returns the name of the local place as str.  This is what Photos displays in the Info window.  **Note** Photos 5 uses a different algorithm to determine the name than earlier versions which means the same Photo may have a different place name in Photos 4 and Photos 5. `PhotoInfo.name` will return the name Photos would have shown depending on the version of the library being processed.  In Photos 5, the place name is generally more detailed than in earlier versions of Photos.
+
+For example, I have photo in my library that under Photos 4, has place name of "‎⁨Mayfair Shopping Centre⁩, ⁨Victoria⁩, ⁨Canada⁩" and under Photos 5 the same photo has place name of "Mayfair⁩, ⁨Vancouver Island⁩, ⁨Victoria⁩, ⁨British Columbia⁩, ⁨Canada⁩".
 
 Returns `None` if photo does not contain a name.
 
 #### `names`
-Returns a list of place names in ascending order by area, starting with the smallest area (most local) to largest area (least local).  For example:
+Returns a `PlaceNames` namedtuple with the following fields.  Each field is a list with zero or more values, sorted by area in ascending order.  E.g. `names.area_of_interest` could be ['Gulf Islands National Seashore', 'Santa Rosa Island'], ["Knott's Berry Farm"], or [] if `area_of_interest` not defined.  The value shown in Photos is the first value in the list. With the exception of `body_of_water` each of these field corresponds to an attribute of a [CLPlacemark](https://developer.apple.com/documentation/corelocation/clplacemark) object.  **Note** The `PlaceNames` namedtuple contains reserved fields not listed below (see implementation for details), thus it should be referenced only by name (e.g. `names.city`) and not by index.
 
-        ["2038 18th St NW",
-        "Adams Morgan",
-        "Washington",
-        "Washington",
-        "Washington",
-        "District of Columbia",
-        "United States"]
+- `country`; the name of the country associated with the placemark.
+- `state_province`; administrativeArea, The state or province associated with the placemark.
+- `sub_administrative_area`; additional administrative area information for the placemark.
+- `city`; locality; the city associated with the placemark.
+- `additional_city_info`; subLocality, Additional city-level information for the placemark.
+- `ocean`; the name of the ocean associated with the placemark.
+- `area_of_interest`; areasOfInterest, The relevant areas of interest associated with the placemark.
+- `inland_water`; the name of the inland water body associated with the placemark.
+- `region`; the geographic region associated with the placemark.
+- `sub_throughfare`; additional street-level information for the placemark.
+- `postal_code`; the postal code associated with the placemark.
+- `street_address`; throughfare, The street address associated with the placemark.
+- `body_of_water`; in Photos 4, any body of water; in Photos 5 contains the union of ocean and inland_water
 
-`names[0]` will always be the most local (e.g. street address) component and `names[-1]` will always be the least local which is almost always the country name.
+**Note**: In Photos <= 4.0, only the following fields are defined; all others are set to empty list:
 
-**Note**: names may contain duplicates as in above.  The data is returned exactly as it is stored by Photos.  
+- `country`
+- `state_province`
+- `sub_administrative_area`
+- `city` 
+- `additional_city_info`
+- `area_of_interest`
+- `body_of_water`
 
 #### `country_code`
 Returns the country_code of place, for example "GB".  Returns `None` if PhotoInfo contains no country code.
@@ -868,15 +954,15 @@ Returns the full postal address as a string if defined, otherwise `None`.
 For example: "2038 18th St NW, Washington, DC  20009, United States"
 
 #### `address`:
-Returns a `PostalAddress` tuple with details of the postal address containing the following fields:
-- city
-- country
-- postal_code
-- state
-- street
-- sub_administrative_area
-- sub_locality
-- iso_country_code
+Returns a `PostalAddress` namedtuple with details of the postal address containing the following fields:
+- `city`
+- `country`
+- `postal_code`
+- `state`
+- `street`
+- `sub_administrative_area`
+- `sub_locality`
+- `iso_country_code`
 
 For example:
 ```python
@@ -886,36 +972,99 @@ PostalAddress(street='3700 Wailea Alanui Dr', sub_locality=None, city='Kihei', s
 '96753'
 ```
 
+### Template Functions
+
+There is a simple template system used by the command line client to specify the output directory using a template.  The following are available in `osxphotos.template`.
+
+#### `render_filepath_template(template, photo, none_str="_")`
+Render template string for photo.  none_str is used if template substitution results in None value and no default specified. 
+- `template`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None.  This is useful for values which are not always present, for example reverse geolocation data.
+- `photo`: a [PhotoInfo](#photoinfo) object
+- `none_str`: optional str to use as substitution when template value is None and no default specified in the template string.  default is "_".
+
+Returns a tuple of (rendered, unmatched) where rendered is the rendered template string with all substitutions made and unmatched is a list of any strings that resembled a template substitution but did not match a known substitution. E.g. strings in the form "{foo}".
+
+e.g. `render_filepath_template("{created.year}/{foo}", photo)` would return `("2020/{foo}",["{foo}"])`
+
+| Substitution | Description |
+|--------------|-------------|
+|{name}|Filename of the photo|
+|{original_name}|Photo's original filename when imported to Photos|
+|{title}|Title of the photo|
+|{descr}|Description of the photo|
+|{created.date}|Photo's creation date in ISO format, e.g. '2020-03-22'|
+|{created.year}|4-digit year of file creation time|
+|{created.yy}|2-digit year of file creation time|
+|{created.mm}|2-digit month of the file creation time (zero padded)|
+|{created.month}|Month name in user's locale of the file creation time|
+|{created.mon}|Month abbreviation in the user's locale of the file creation time|
+|{created.doy}|3-digit day of year (e.g Julian day) of file creation time, starting from 1 (zero padded)|
+|{modified.date}|Photo's modification date in ISO format, e.g. '2020-03-22'|
+|{modified.year}|4-digit year of file modification time|
+|{modified.yy}|2-digit year of file modification time|
+|{modified.mm}|2-digit month of the file modification time (zero padded)|
+|{modified.month}|Month name in user's locale of the file modification time|
+|{modified.mon}|Month abbreviation in the user's locale of the file modification time|
+|{modified.doy}|3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)|
+|{place.name}|Place name from the photo's reverse geolocation data, as displayed in Photos|
+|{place.country_code}|The ISO country code from the photo's reverse geolocationo data|
+|{place.name.country}|Country name from the photo's reverse geolocation data|
+|{place.name.state_province}|State or province name from the photo's reverse geolocation data|
+|{place.name.city}|City or locality name from the photo's reverse geolocation data|
+|{place.name.area_of_interest}|Area of interest name (e.g. landmark or public place) from the photo's reverse geolocation data|
+|{place.address}|Postal address from the photo's reverse geolocation data, e.g. '2007 18th St NW, Washington, DC 20009, United States'|
+|{place.address.street}|Street part of the postal address, e.g. '2007 18th St NW'|
+|{place.address.city}|City part of the postal address, e.g. 'Washington'|
+|{place.address.state_province}|State/province part of the postal address, e.g. 'DC'|
+|{place.address.postal_code}|Postal code part of the postal address, e.g. '20009'|
+|{place.address.country}|Country name of the postal address, e.g. 'United States'|
+|{place.address.country_code}|ISO country code of the postal address, e.g. 'US'|
+
+#### `DateTimeFormatter(dt)`
+Class that provides easy access to formatted datetime values.
+- `dt`: a datetime.datetime object
+
+Returnes `DateTimeFormater` class.
+
+Has the following properties:
+- `date`: Date in ISO format without timezone, e.g. "2020-03-04"
+- `year`: 4-digit year
+- `yy`: 2-digit year
+- `month`: month name in user's locale
+- `mon`: month abbreviation in user's locale
+- `mm`: 2-digit month
+- `doy`: 3-digit day of year (e.g. Julian day)
+
 ### Utility Functions
 
 The following functions are located in osxphotos.utils
 
-#### ```get_system_library_path()```
+#### `get_system_library_path()`
 
 **MacOS 10.15 Only** Returns path to System Photo Library as string.  On MacOS version < 10.15, returns None.
 
-#### ```get_last_library_path()```
+#### `get_last_library_path()`
 
 Returns path to last opened Photo Library as string.  
 
-#### ```list_photo_libraries()```
+#### `list_photo_libraries()`
 
 Returns list of Photos libraries found on the system.  **Note**: On MacOS 10.15, this appears to list all libraries. On older systems, it may not find some libraries if they are not located in ~/Pictures.  Provided for convenience but do not rely on this to find all libraries on the system.
 
-#### ```dd_to_dms_str(lat, lon)```
+#### `dd_to_dms_str(lat, lon)`
 Convert latitude, longitude in degrees to degrees, minutes, seconds as string.
-lat: latitude in degrees
-lon: longitude in degrees
+- `lat`: latitude in degrees
+- `lon`: longitude in degrees
 returns: string tuple in format ("51 deg 30' 12.86\\" N", "0 deg 7' 54.50\\" W")
 This is the same format used by exiftool's json format.
 
-#### ```create_path_by_date(dest, dt)```
+#### `create_path_by_date(dest, dt)`
 Creates a path in dest folder in form dest/YYYY/MM/DD/
-dest: valid path as str
-dt: datetime.timetuple() object
+- `dest`: valid path as str
+- `dt`: datetime.timetuple() object
 Checks to see if path exists, if it does, do nothing and return path. If path does not exist, creates it and returns path.  Useful for exporting photos to a date-based folder structure.
 
-### Examples
+## Examples
 
 ```python
 import osxphotos
@@ -969,7 +1118,7 @@ if __name__ == "__main__":
 
 ## Related Projects
 
-- [photosmeta](https://github.com/rhettbull/photosmeta): uses osxphotos and [exiftool](https://exiftool.org/) to apply metadata from Photos as exif data in the photo files.  Can also export photos while preserving metadata and also apply Photos keywords as spotlight tags to make it easier to search for photos using spotlight.  This is mostly made obsolete by osxphotos.  The one feature that photosmeta has that osxphotos does not is ability to update the metadata of the actual photo files in the Photos library without exporting them. (Use with caution!)
+- [rhettbull/photosmeta](https://github.com/rhettbull/photosmeta): uses osxphotos and [exiftool](https://exiftool.org/) to apply metadata from Photos as exif data in the photo files.  Can also export photos while preserving metadata and also apply Photos keywords as spotlight tags to make it easier to search for photos using spotlight.  This is mostly made obsolete by osxphotos.  The one feature that photosmeta has that osxphotos does not is ability to update the metadata of the actual photo files in the Photos library without exporting them. (Use with caution!)
 - [patrikhson/photo-export](https://github.com/patrikhson/photo-export): Exports older versions of Photos databases.  Provided the inspiration for osxphotos.
 - [orangeturtle739/photos-export](https://github.com/orangeturtle739/photos-export): Set of scripts to export Photos libraries.
 - [ndbroadbent/icloud_photos_downloader](https://github.com/ndbroadbent/icloud_photos_downloader): Download photos from iCloud.  Currently unmaintained.
@@ -1001,6 +1150,7 @@ Apple does provide a framework ([PhotoKit](https://developer.apple.com/documenta
 - [Click](https://pypi.org/project/click/)
 - [Mako](https://www.makotemplates.org/)
 - [bpylist2](https://pypi.org/project/bpylist2/)
+- [pathvalidate](https://pypi.org/project/pathvalidate/)
 
 ## Acknowledgements
 This project was originally inspired by [photo-export](https://github.com/patrikhson/photo-export) by Patrick Fältström,  Copyright (c) 2015 Patrik Fältström paf@frobbit.se

examples/export_by_album.py

@@ -0,0 +1,75 @@
+""" Export all photos to specified directory using album names as folders
+    If file has been edited, also export the edited version, 
+    otherwise, export the original version 
+    This will result in duplicate photos if photo is in more than album """
+
+import os.path
+import pathlib
+import sys
+
+import click
+from pathvalidate import is_valid_filepath, sanitize_filepath
+
+import osxphotos
+
+
+@click.command()
+@click.argument("export_path", type=click.Path(exists=True))
+@click.option(
+    "--default-album",
+    help="Default folder for photos with no album. Defaults to 'unfiled'",
+    default="unfiled",
+)
+@click.option(
+    "--library-path",
+    help="Path to Photos library, default to last used library",
+    default=None,
+)
+def export(export_path, default_album, library_path):
+    export_path = os.path.expanduser(export_path)
+    library_path = os.path.expanduser(library_path) if library_path else None
+
+    if library_path is not None:
+        photosdb = osxphotos.PhotosDB(library_path)
+    else:
+        photosdb = osxphotos.PhotosDB()
+
+    photos = photosdb.photos()
+
+    for p in photos:
+        if not p.ismissing:
+            albums = p.albums
+            if not albums:
+                albums = [default_album]
+            for album in albums:
+                click.echo(f"exporting {p.filename} in album {album}")
+
+                # make sure no invalid characters in destination path (could be in album name)
+                album_name = sanitize_filepath(album, platform="auto")
+
+                # create destination folder, if necessary, based on album name
+                dest_dir = os.path.join(export_path, album_name)
+
+                # verify path is a valid path
+                if not is_valid_filepath(dest_dir, platform="auto"):
+                    sys.exit(f"Invalid filepath {dest_dir}")
+
+                # create destination dir if needed
+                if not os.path.isdir(dest_dir):
+                    os.makedirs(dest_dir)
+
+                # export the photo
+                if p.hasadjustments:
+                    # export edited version
+                    exported = p.export(dest_dir, edited=True)
+                    edited_name = pathlib.Path(p.path_edited).name
+                    click.echo(f"Exported {edited_name} to {exported}")
+                # export unedited version
+                exported = p.export(dest_dir)
+                click.echo(f"Exported {p.filename} to {exported}")
+        else:
+            click.echo(f"Skipping missing photo: {p.filename}")
+
+
+if __name__ == "__main__":
+    export()  # pylint: disable=no-value-for-parameter

osxphotos/__main__.py

@@ -21,7 +21,7 @@ import osxphotos
 from ._constants import _EXIF_TOOL_URL, _PHOTOS_5_VERSION
 from ._version import __version__
 from .exiftool import get_exiftool_path
-from .template import render_filename_template, TEMPLATE_SUBSTITUTIONS
+from .template import render_filepath_template, TEMPLATE_SUBSTITUTIONS
 from .utils import _copy_file, create_path_by_date
 
 
@@ -81,11 +81,11 @@ class ExportCommand(click.Command):
         formatter.write_text(
             "With the --directory option, you may specify a template for the "
             + "export directory.  This directory will be appended to the export path specified "
-            + " in the export DEST argument to export.  For example, if template is "
+            + "in the export DEST argument to export.  For example, if template is "
             + "'{created.year}/{created.month}', and export desitnation DEST is "
             + "'/Users/maria/Pictures/export', "
-            + " the actual export directory for a photo would be '/Users/maria/Pictures/export/2020/March' "
-            + " if the photo was created in March 2020. "
+            + "the actual export directory for a photo would be '/Users/maria/Pictures/export/2020/March' "
+            + "if the photo was created in March 2020. "
         )
         formatter.write("\n")
         formatter.write_text(
@@ -104,16 +104,22 @@ class ExportCommand(click.Command):
         )
         formatter.write("\n")
         formatter.write_text(
-            "In the current implementation, substitutions which have no value "
-            + "will be replaced by '_', "
-            + "for example, your template looked like '{created.year}/{place.address}' "
+            "You may specify an optional default value to use if the substitution does not contain a value "
+            + "(e.g. the value is null) "
+            + "by specifying the default value after a ',' in the template string: "
+            + "for example, if template is '{created.year}/{place.address,'NO_ADDRESS'}' "
             + "but there was no address associated with the photo, the resulting output would be: "
-            + "'2020/_/photoname.jpg' "
+            + "'2020/NO_ADDRESS/photoname.jpg'. "
+            + "If specified, the default value may not contain a brace symbol ('{' or '}')."
         )
         formatter.write("\n")
         formatter.write_text(
-            "I plan to add the option to specify the value to be used for missing "
-            + "subsitutions in a future version. I also plan to extend the templating system "
+            "If you do not specify a default value and the template substitution "
+            + "has no value, '_' (underscore) will be used as the default value. For example, in the "
+            + "above example, this would result in '2020/_/photoname.jpg' if address was null"
+        )
+        formatter.write_text(
+            "I plan to eventually extend the templating system "
             + "to the exported filename so you can specify the filename using a template."
         )
 
@@ -817,8 +823,8 @@ def query(
     "--directory",
     metavar="DIRECTORY",
     default=None,
-    help="Optional template for specifying name of output directory.  "
-    "See below for additional details on templating system",
+    help="Optional template for specifying name of output directory in the form '{name,DEFAULT}'. "
+    "See below for additional details on templating system.",
 )
 @DB_ARGUMENT
 @click.argument("dest", nargs=1, type=click.Path(exists=True))
@@ -1403,13 +1409,13 @@ def export_photo(
         date_created = photo.date.timetuple()
         dest = create_path_by_date(dest, date_created)
     elif directory:
-        dirname, unmatched = render_filename_template(directory, photo)
+        dirname, unmatched = render_filepath_template(directory, photo)
         if unmatched:
             click.echo(
                 f"Possible unmatched substitution in template: {unmatched}", err=True
             )
-        dirname = sanitize_filepath(dirname)
-        if not is_valid_filepath(dirname):
+        dirname = sanitize_filepath(dirname, platform="auto")
+        if not is_valid_filepath(dirname, platform="auto"):
             raise ValueError(f"Invalid file path: {dirname}")
         dest = os.path.join(dest, dirname)
         if not os.path.isdir(dest):

osxphotos/_version.py

@@ -1,3 +1,3 @@
 """ version info """
 
-__version__ = "0.23.3"
+__version__ = "0.24.2"

osxphotos/exiftool.py

@@ -10,7 +10,7 @@ import logging
 import os
 import subprocess
 import sys
-from functools import lru_cache
+from functools import lru_cache # pylint: disable=syntax-error
 
 from .utils import _debug
 

osxphotos/photoinfo.py

@@ -116,8 +116,6 @@ class PhotoInfo:
             )
             return photopath
 
-        # if self._info["masterFingerprint"]:
-        # if masterFingerprint is not null, path appears to be valid
         if self._info["directory"].startswith("/"):
             photopath = os.path.join(self._info["directory"], self._info["filename"])
         else:
@@ -126,13 +124,6 @@ class PhotoInfo:
             )
         return photopath
 
-        # if all else fails, photopath = None
-        # photopath = None
-        # logging.debug(
-        #     f"WARNING: photopath None, masterFingerprint null, not shared {pformat(self._info)}"
-        # )
-        # return photopath
-
     @property
     def path_edited(self):
         """ absolute path on disk of the edited picture """
@@ -492,7 +483,7 @@ class PhotoInfo:
 
     @property
     def place(self):
-        """ If Photos version >= 5, returns PlaceInfo object containing reverse geolocation info """
+        """ Returns PlaceInfo object containing reverse geolocation info """
 
         # implementation note: doesn't create the PlaceInfo object until requested
         # then memoizes the object in self._place to avoid recreating the object
@@ -500,7 +491,7 @@ class PhotoInfo:
         if self._db._db_version < _PHOTOS_5_VERSION:
             try:
                 return self._place  # pylint: disable=access-member-before-definition
-            except:
+            except AttributeError:
                 if self._info["placeNames"]:
                     self._place = PlaceInfo4(
                         self._info["placeNames"], self._info["countryCode"]

osxphotos/photosdb.py

@@ -395,18 +395,6 @@ class PhotosDB:
 
         return dest_path
 
-    # def _open_sql_file(self, fname):
-    #     """ opens sqlite file fname in read-only mode
-    #         returns tuple of (connection, cursor) """
-    #     try:
-    #         conn = sqlite3.connect(
-    #             f"{pathlib.Path(fname).as_uri()}?mode=ro", timeout=1, uri=True
-    #         )
-    #         c = conn.cursor()
-    #     except sqlite3.Error as e:
-    #         sys.exit(f"An error occurred opening sqlite file: {e.args[0]} {fname}")
-    #     return (conn, c)
-
     def _get_db_version(self):
         """ gets the Photos DB version from LiGlobals table """
         """ returns the version as str"""
@@ -844,21 +832,26 @@ class PhotosDB:
         countries = {code[0]: code[1] for code in country_codes}
         self._db_countries = countries
 
-        # save existing row_factory
-        old_row_factory = c.row_factory
-
-        # want only the list of values, not a list of tuples
-        c.row_factory = lambda cursor, row: row[0]
+        # get the place data
+        place_data = c.execute(
+            "SELECT modelID, defaultName, type, area " "FROM RKPlace "
+        ).fetchall()
+        places = {p[0]: p for p in place_data}
+        self._db_places = places
 
         for uuid in self._dbphotos:
             # get placeId which is then used to lookup defaultName
-            place_ids = c.execute(
+            place_ids_query = c.execute(
                 "SELECT placeId "
                 "FROM RKPlaceForVersion "
                 f"WHERE versionId = '{self._dbphotos[uuid]['modelID']}'"
-            ).fetchall()
-            self._dbphotos[uuid]["placeIDs"] = place_ids
-            country_code = [countries[x] for x in place_ids if x in countries]
+            )
+            
+            place_ids = [id[0] for id in place_ids_query.fetchall()]
+            self._dbphotos[uuid]["placeIDs"]  = place_ids
+            country_code = [
+                countries[x] for x in place_ids if x in countries
+            ]
             if len(country_code) > 1:
                 logging.warning(f"Found more than one country code for uuid: {uuid}")
 
@@ -867,19 +860,19 @@ class PhotosDB:
             else:
                 self._dbphotos[uuid]["countryCode"] = None
 
-            place_names = c.execute(
-                "SELECT DISTINCT defaultName AS name "
-                "FROM RKPlace "
-                f"WHERE modelId IN({','.join(map(str,place_ids))}) "
-                "ORDER BY area ASC "
-            ).fetchall()
+            # get the place info that matches the RKPlace modelIDs for this photo 
+            # (place_ids), sort by area (element 3 of the place_data tuple in places)
+            place_names = [
+                pname
+                for pname in sorted(
+                    [places[p] for p in places if p in place_ids],
+                    key=lambda place: place[3],
+                )
+            ]
 
             self._dbphotos[uuid]["placeNames"] = place_names
             self._dbphotos[uuid]["reverse_geolocation"] = None  # Photos 5
 
-        # restore row_factory
-        c.row_factory = old_row_factory
-
         # build album_titles dictionary
         for album_id in self._dbalbum_details:
             title = self._dbalbum_details[album_id]["title"]

osxphotos/placeinfo.py

@@ -1,6 +1,9 @@
 """ 
     PlaceInfo class
     Provides reverse geolocation info for photos 
+    
+    See https://developer.apple.com/documentation/corelocation/clplacemark
+    for additional documentation on reverse geolocation data
 """
 from abc import ABC, abstractmethod
 from collections import namedtuple  # pylint: disable=syntax-error
@@ -16,13 +19,45 @@ PostalAddress = namedtuple(
         "sub_locality",
         "city",
         "sub_administrative_area",
-        "state",
+        "state_province",
         "postal_code",
         "country",
         "iso_country_code",
     ],
 )
 
+# PlaceNames tuple returned by PlaceInfo.names
+# order of fields 0 - 17 is mapped to placeType value in
+# PLRevGeoLocationInfo.mapInfo.sortedPlaceInfos
+# field 18 is combined bodies of water (ocean + inland_water)
+# and maps to Photos <= 4, RKPlace.type == 44
+# (Photos <= 4 doesn't have ocean or inland_water types)
+# The fields named "field0", etc. appear to be unused
+PlaceNames = namedtuple(
+    "PlaceNames",
+    [
+        "field0",
+        "country",  # The name of the country associated with the placemark.
+        "state_province",  # administrativeArea, The state or province associated with the placemark.
+        "sub_administrative_area",  # Additional administrative area information for the placemark.
+        "city",  # locality, The city associated with the placemark.
+        "field5",
+        "additional_city_info",  # subLocality, Additional city-level information for the placemark.
+        "ocean",  # The name of the ocean associated with the placemark.
+        "area_of_interest",  # areasOfInterest, The relevant areas of interest associated with the placemark.
+        "inland_water",  # The name of the inland water body associated with the placemark.
+        "field10",
+        "region",  # The geographic region associated with the placemark.
+        "sub_throughfare",  # Additional street-level information for the placemark.
+        "field13",
+        "postal_code",  # The postal code associated with the placemark.
+        "field15",
+        "field16",
+        "street_address",  # throughfare, The street address associated with the placemark.
+        "body_of_water",  # RKPlace.type == 44, appears to be any body of water (ocean or inland)
+    ],
+)
+
 # The following classes represent Photo Library Reverse Geolocation Info as stored
 # in ZADDITIONALASSETATTRIBUTES.ZREVERSELOCATIONDATA
 # These classes are used by bpylist.archiver to unarchive the serialized objects
@@ -323,9 +358,18 @@ class PlaceInfo4(PlaceInfo):
     """ Reverse geolocation place info for a photo (Photos <= 4) """
 
     def __init__(self, place_names, country_code):
-        """ place_names: list of place names in ascending order by area """
+        """ place_names: list of place name tuples in ascending order by area 
+            tuple fields are: modelID, place name, place type, area, e.g.
+            [(5, "St James's Park", 45, 0), 
+            (4, 'Westminster', 16, 22097376), 
+            (3, 'London', 4, 1596146816), 
+            (2, 'England', 2, 180406091776), 
+            (1, 'United Kingdom', 1, 414681432064)] 
+            country_code: two letter country code for the country
+        """
         self._place_names = place_names
         self._country_code = country_code
+        self._process_place_info()
 
     @property
     def address_str(self):
@@ -341,11 +385,11 @@ class PlaceInfo4(PlaceInfo):
 
     @property
     def name(self):
-        return self._place_names[0]
+        return self._name
 
     @property
     def names(self):
-        return self._place_names
+        return self._names
 
     @property
     def address(self):
@@ -360,6 +404,83 @@ class PlaceInfo4(PlaceInfo):
                 and self._country_code == other._country_code
             )
 
+    def _process_place_info(self):
+        """ Process place_names to set self._name and self._names """
+        places = self._place_names
+
+        # build a dictionary where key is placetype
+        places_dict = {}
+        for p in places:
+            # places in format:
+            # [(5, "St James's Park", 45, 0), ]
+            #   0: modelID
+            #   1: name
+            #   2: type
+            #   3: area
+            try:
+                places_dict[p[2]].append((p[1], p[3]))
+            except KeyError:
+                places_dict[p[2]] = [(p[1], p[3])]
+
+        # build list to populate PlaceNames tuple
+        # initialize with empty lists for each field in PlaceNames
+        place_info = [[]] * 19
+
+        # add the place names sorted by area (ascending)
+        # in Photos <=4, possible place type values are:
+        # 45: areasOfInterest (The relevant areas of interest associated with the placemark.)
+        # 44: body of water (includes both inlandWater and ocean)
+        # 43: subLocality (Additional city-level information for the placemark.
+        # 16: locality (The city associated with the placemark.)
+        # 4: subAdministrativeArea (Additional administrative area information for the placemark.)
+        # 2: administrativeArea (The state or province associated with the placemark.)
+        # 1: country
+        # mapping = mapping from PlaceNames to field in places_dict
+        # PlaceNames fields map to the placeType value in Photos5 (0..17)
+        # but place type in Photos <=4 has different values
+        # hence (3, 4) means PlaceNames[3] = places_dict[4] (sub_administrative_area)
+        mapping = [(1, 1), (2, 2), (3, 4), (4, 16), (18, 44), (8, 45)]
+        for field5, field4 in mapping:
+            try:
+                place_info[field5] = [
+                    p[0]
+                    for p in sorted(places_dict[field4], key=lambda place: place[1])
+                ]
+            except KeyError:
+                pass
+
+        place_names = PlaceNames(*place_info)
+        self._names = place_names
+
+        # build the name as it appears in Photos
+        # the length of the name is at most 3 fields and appears to be based on available
+        # reverse geolocation data in the following order (left to right, joined by ',')
+        # always has country if available then either area of interest and city OR
+        # city and state
+        # e.g. 4, 2, 1 OR 8, 4, 1
+        # 8 (45): area_of_interest
+        # 4 (16): locality / city
+        # 2 (2): administrative area (state/province)
+        # 1 (1): country
+        name_list = []
+        if place_names[8]:
+            name_list.append(place_names[8][0])
+            if place_names[4]:
+                name_list.append(place_names[4][0])
+        elif place_names[4]:
+            name_list.append(place_names[4][0])
+            if place_names[2]:
+                name_list.append(place_names[2][0])
+        elif place_names[2]:
+            name_list.append(place_names[2][0])
+
+        # add country
+        if place_names[1]:
+            name_list.append(place_names[1][0])
+
+        name = ", ".join(name_list)
+        self._name = name if name != "" else None
+
     def __ne__(self, other):
         return not self.__eq__(other)
 
@@ -382,6 +503,7 @@ class PlaceInfo5(PlaceInfo):
         self._bplist = revgeoloc_bplist
         # todo: check for None?
         self._plrevgeoloc = archiver.unarchive(revgeoloc_bplist)
+        self._process_place_info()
 
     @property
     def address_str(self):
@@ -401,22 +523,12 @@ class PlaceInfo5(PlaceInfo):
     @property
     def name(self):
         """ returns local place name """
-        name = (
-            self._plrevgeoloc.mapItem.sortedPlaceInfos[0].name
-            if self._plrevgeoloc.mapItem.sortedPlaceInfos
-            else None
-        )
-        return name
+        return self._name
 
     @property
     def names(self):
-        """ returns list of all place names in reverse order by area
-            e.g. most local is at index 0, least local (usually country) is at index -1 """
-        names = []
-        # todo: strip duplicates
-        for name in self._plrevgeoloc.mapItem.sortedPlaceInfos:
-            names.append(name.name)
-        return names
+        """ returns PlaceNames tuple with detailed reverse geolocation place names """
+        return self._names
 
     @property
     def address(self):
@@ -426,13 +538,72 @@ class PlaceInfo5(PlaceInfo):
             sub_locality=addr._subLocality,
             city=addr._city,
             sub_administrative_area=addr._subAdministrativeArea,
-            state=addr._state,
+            state_province=addr._state,
             postal_code=addr._postalCode,
             country=addr._country,
             iso_country_code=addr._ISOCountryCode,
         )
         return address
 
+    def _process_place_info(self):
+        """ Process sortedPlaceInfos to set self._name and self._names """
+        places = self._plrevgeoloc.mapItem.sortedPlaceInfos
+
+        # build a dictionary where key is placetype
+        places_dict = {}
+        for p in places:
+            try:
+                places_dict[p.placeType].append((p.name, p.area))
+            except KeyError:
+                places_dict[p.placeType] = [(p.name, p.area)]
+
+        # build list to populate PlaceNames tuple
+        place_info = []
+        for field in range(18):
+            try:
+                # add the place names sorted by area (ascending)
+                place_info.append(
+                    [
+                        p[0]
+                        for p in sorted(places_dict[field], key=lambda place: place[1])
+                    ]
+                )
+            except:
+                place_info.append([])
+
+        # fill in body_of_water for compatibility with Photos <= 4
+        place_info.append(place_info[7] + place_info[9])
+
+        place_names = PlaceNames(*place_info)
+        self._names = place_names
+
+        # build the name as it appears in Photos
+        # the length of the name is variable and appears to be based on available
+        # reverse geolocation data in the following order (left to right, joined by ',')
+        # 8: area_of_interest
+        # 11: region (I've only seen this applied to islands)
+        # 4: locality / city
+        # 2: administrative area (state/province)
+        # 1: country
+        # 9: inland_water
+        # 7: ocean
+        name = ", ".join(
+            [
+                p[0]
+                for p in [
+                    place_names[8],  # area of interest
+                    place_names[11],  # region (I've only seen this applied to islands)
+                    place_names[4],  # locality / city
+                    place_names[2],  # administrative area (state/province)
+                    place_names[1],  # country
+                    place_names[9],  # inland_water
+                    place_names[7],  # ocean
+                ]
+                if p and p[0]
+            ]
+        )
+        self._name = name if name != "" else None
+
     def __eq__(self, other):
         if not isinstance(other, type(self)):
             return False

osxphotos/template.py

@@ -24,137 +24,251 @@ TEMPLATE_SUBSTITUTIONS = {
     "{modified.month}": "Month name in user's locale of the file modification time",
     "{modified.mon}": "Month abbreviation in the user's locale of the file modification time",
     "{modified.doy}": "3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)",
-    "{place.name}": "Place name from the photo's reverse geolocation data",
-    "{place.names}": "list of place names from the photo's reverse geolocation data, joined with '_', for example, '18th St NW_Washington_DC_United States'",
+    "{place.name}": "Place name from the photo's reverse geolocation data, as displayed in Photos",
+    "{place.country_code}": "The ISO country code from the photo's reverse geolocationo data",
+    "{place.name.country}": "Country name from the photo's reverse geolocation data",
+    "{place.name.state_province}": "State or province name from the photo's reverse geolocation data",
+    "{place.name.city}": "City or locality name from the photo's reverse geolocation data",
+    "{place.name.area_of_interest}": "Area of interest name (e.g. landmark or public place) from the photo's reverse geolocation data",
     "{place.address}": "Postal address from the photo's reverse geolocation data, e.g. '2007 18th St NW, Washington, DC 20009, United States'",
-    "{place.street}": "Street part of the postal address, e.g. '2007 18th St NW'",
-    "{place.city}": "City part of the postal address, e.g. 'Washington'",
-    "{place.state}": "State part of the postal address, e.g. 'DC'",
-    "{place.postal_code}": "Postal code part of the postal address, e.g. '20009'",
-    "{place.country}": "Country name of the postal code, e.g. 'United States'",
-    "{place.country_code}": "ISO country code of the postal address, e.g. 'US'",
+    "{place.address.street}": "Street part of the postal address, e.g. '2007 18th St NW'",
+    "{place.address.city}": "City part of the postal address, e.g. 'Washington'",
+    "{place.address.state_province}": "State/province part of the postal address, e.g. 'DC'",
+    "{place.address.postal_code}": "Postal code part of the postal address, e.g. '20009'",
+    "{place.address.country}": "Country name of the postal address, e.g. 'United States'",
+    "{place.address.country_code}": "ISO country code of the postal address, e.g. 'US'",
 }
 
 
-def render_filename_template(
+def get_template_value(lookup, photo):
+    """ lookup: value to find a match for
+        photo: PhotoInfo object whose data will be used for value substitutions
+        returns: either the matching template value (which may be None)
+        raises: KeyError if no rule exists for lookup """
+
+    # must be a valid keyword
+    if lookup == "name":
+        return pathlib.Path(photo.filename).stem
+
+    if lookup == "original_name":
+        return pathlib.Path(photo.original_filename).stem
+
+    if lookup == "title":
+        return photo.title
+
+    if lookup == "descr":
+        return photo.description
+
+    if lookup == "created.date":
+        return DateTimeFormatter(photo.date).date
+
+    if lookup == "created.year":
+        return DateTimeFormatter(photo.date).year
+
+    if lookup == "created.yy":
+        return DateTimeFormatter(photo.date).yy
+
+    if lookup == "created.mm":
+        return DateTimeFormatter(photo.date).mm
+
+    if lookup == "created.month":
+        return DateTimeFormatter(photo.date).month
+
+    if lookup == "created.mon":
+        return DateTimeFormatter(photo.date).mon
+
+    if lookup == "created.doy":
+        return DateTimeFormatter(photo.date).doy
+
+    if lookup == "modified.date":
+        return (
+            DateTimeFormatter(photo.date_modified).date if photo.date_modified else None
+        )
+
+    if lookup == "modified.year":
+        return (
+            DateTimeFormatter(photo.date_modified).year if photo.date_modified else None
+        )
+
+    if lookup == "modified.yy":
+        return (
+            DateTimeFormatter(photo.date_modified).yy if photo.date_modified else None
+        )
+
+    if lookup == "modified.mm":
+        return (
+            DateTimeFormatter(photo.date_modified).mm if photo.date_modified else None
+        )
+
+    if lookup == "modified.month":
+        return (
+            DateTimeFormatter(photo.date_modified).month
+            if photo.date_modified
+            else None
+        )
+
+    if lookup == "modified.mon":
+        return (
+            DateTimeFormatter(photo.date_modified).mon if photo.date_modified else None
+        )
+
+    if lookup == "modified.doy":
+        return (
+            DateTimeFormatter(photo.date_modified).doy if photo.date_modified else None
+        )
+
+    if lookup == "place.name":
+        return photo.place.name if photo.place else None
+
+    if lookup == "place.country_code":
+        return photo.place.country_code if photo.place else None
+
+    if lookup == "place.name.country":
+        return (
+            photo.place.names.country[0]
+            if photo.place and photo.place.names.country
+            else None
+        )
+
+    if lookup == "place.name.state_province":
+        return (
+            photo.place.names.state_province[0]
+            if photo.place and photo.place.names.state_province
+            else None
+        )
+
+    if lookup == "place.name.city":
+        return (
+            photo.place.names.city[0]
+            if photo.place and photo.place.names.city
+            else None
+        )
+
+    if lookup == "place.name.area_of_interest":
+        return (
+            photo.place.names.area_of_interest[0]
+            if photo.place and photo.place.names.area_of_interest
+            else None
+        )
+
+    if lookup == "place.address":
+        return (
+            photo.place.address_str if photo.place and photo.place.address_str else None
+        )
+
+    if lookup == "place.address.street":
+        return (
+            photo.place.address.street
+            if photo.place and photo.place.address.street
+            else None
+        )
+
+    if lookup == "place.address.city":
+        return (
+            photo.place.address.city
+            if photo.place and photo.place.address.city
+            else None
+        )
+
+    if lookup == "place.address.state_province":
+        return (
+            photo.place.address.state_province
+            if photo.place and photo.place.address.state_province
+            else None
+        )
+
+    if lookup == "place.address.postal_code":
+        return (
+            photo.place.address.postal_code
+            if photo.place and photo.place.address.postal_code
+            else None
+        )
+
+    if lookup == "place.address.country":
+        return (
+            photo.place.address.country
+            if photo.place and photo.place.address.country
+            else None
+        )
+
+    if lookup == "place.address.country_code":
+        return (
+            photo.place.address.iso_country_code
+            if photo.place and photo.place.address.iso_country_code
+            else None
+        )
+
+    # if here, didn't get a match
+    raise KeyError(f"No rule for processing {lookup}")
+
+
+def render_filepath_template(
     template: str, photo: PhotoInfo, none_str: str = "_"
 ) -> Tuple[str, list]:
     """ render a filename or directory template """
 
+    # pylint: disable=anomalous-backslash-in-string
+    regex = r"""(?<!\\)\{([^\\,}]+)(,{0,1}(([\w\-. ]+))?)\}"""
+
+    # pylint: disable=anomalous-backslash-in-string
+    unmatched_regex = r"(?<!\\)(\{[^\\,}]+\})"
+
+    # Explanation for regex:
+    # (?<!\\) Negative Lookbehind to skip escaped braces
+    #     assert regex following does not match "\" preceeding "{"
+    # \{ Match the opening brace
+    # 1st Capturing Group ([^\\,}]+)  Don't match "\", ",", or "}"
+    # 2nd Capturing Group (,?(([\w\-. ]+))?)
+    #     ,{0,1} optional ","
+    # 3rd Capturing Group (([\w\-. ]+))?
+    #     Matches the comma and any word characters after
+    # 4th Capturing Group ([\w\-. ]+)
+    #     Matches just the characters after the comma
+    # \} Matches the closing brace
+
     if type(template) is not str:
         raise TypeError(f"template must be type str, not {type(template)}")
 
     if type(photo) is not PhotoInfo:
         raise TypeError(f"photo must be type osxphotos.PhotoInfo, not {type(photo)}")
 
-    rendered = template
-    original_name = pathlib.Path(photo.original_filename).stem
-    current_name = pathlib.Path(photo.filename).stem
-    created = DateTimeFormatter(photo.date)
-    if photo.date_modified:
-        modified = DateTimeFormatter(photo.date_modified)
-    else:
-        modified = None
+    def make_subst_function(photo, none_str):
+        """ returns: substitution function for use in re.sub """
+        # closure to capture photo, none_str in subst
+        def subst(matchobj):
+            groups = len(matchobj.groups())
+            if groups == 4:
+                try:
+                    val = get_template_value(matchobj.group(1), photo)
+                except KeyError:
+                    return matchobj.group(0)
 
-    # make substitutions
-    rendered = rendered.replace("{name}", current_name)
-    rendered = rendered.replace("{original_name}", original_name)
+                if val is None:
+                    return (
+                        matchobj.group(3) if matchobj.group(3) is not None else none_str
+                    )
+                else:
+                    return val
+            else:
+                raise ValueError(
+                    f"Unexpected number of groups: expected 4, got {groups}"
+                )
 
-    title = photo.title if photo.title is not None else none_str
-    rendered = rendered.replace("{title}", f"{title}")
+        return subst
 
-    descr = photo.description if photo.description is not None else none_str
-    rendered = rendered.replace("{descr}", f"{descr}")
+    subst_func = make_subst_function(photo, none_str)
 
-    rendered = rendered.replace("{created.date}", photo.date.date().isoformat())
-    rendered = rendered.replace("{created.year}", created.year)
-    rendered = rendered.replace("{created.yy}", created.yy)
-    rendered = rendered.replace("{created.mm}", created.mm)
-    rendered = rendered.replace("{created.month}", created.month)
-    rendered = rendered.replace("{created.mon}", created.mon)
-    rendered = rendered.replace("{created.doy}", created.doy)
+    # do the replacements
+    rendered = re.sub(regex, subst_func, template)
 
-    if modified is not None:
-        rendered = rendered.replace(
-            "{modified.date}", photo.date_modified.date().isoformat()
-        )
-        rendered = rendered.replace("{modified.year}", modified.year)
-        rendered = rendered.replace("{modified.yy}", modified.yy)
-        rendered = rendered.replace("{modified.mm}", modified.mm)
-        rendered = rendered.replace("{modified.month}", modified.month)
-        rendered = rendered.replace("{modified.mon}", modified.mon)
-        rendered = rendered.replace("{modified.doy}", modified.doy)
-    else:
-        rendered = rendered.replace("{modified.year}", none_str)
-        rendered = rendered.replace("{modified.yy}", none_str)
-        rendered = rendered.replace("{modified.mm}", none_str)
-        rendered = rendered.replace("{modified.month}", none_str)
-        rendered = rendered.replace("{modified.mon}", none_str)
-        rendered = rendered.replace("{modified.doy}", none_str)
-
-    place_name = photo.place.name if photo.place and photo.place.name else none_str
-    rendered = rendered.replace("{place.name}", place_name)
-
-    place_names = (
-        "_".join(photo.place.names) if photo.place and photo.place.names else none_str
-    )
-    rendered = rendered.replace("{place.names}", place_names)
-
-    address = (
-        photo.place.address_str if photo.place and photo.place.address_str else none_str
-    )
-    rendered = rendered.replace("{place.address}", address)
-
-    street = (
-        photo.place.address.street
-        if photo.place and photo.place.address.street
-        else none_str
-    )
-    rendered = rendered.replace("{place.street}", street)
-
-    city = (
-        photo.place.address.city
-        if photo.place and photo.place.address.city
-        else none_str
-    )
-    rendered = rendered.replace("{place.city}", city)
-
-    state = (
-        photo.place.address.state
-        if photo.place and photo.place.address.state
-        else none_str
-    )
-    rendered = rendered.replace("{place.state}", state)
-
-    postal_code = (
-        photo.place.address.state
-        if photo.place and photo.place.address.postal_code
-        else none_str
-    )
-    rendered = rendered.replace("{place.postal_code}", postal_code)
-
-    country = (
-        photo.place.address.state
-        if photo.place and photo.place.address.country
-        else none_str
-    )
-    rendered = rendered.replace("{place.country}", country)
-
-    country_code = (
-        photo.place.country_code
-        if photo.place and photo.place.country_code
-        else none_str
-    )
-    rendered = rendered.replace("{place.country_code}", country_code)
+    # find any {words} that weren't replaced
+    unmatched = re.findall(unmatched_regex, rendered)
 
     # fix any escaped curly braces
     rendered = re.sub(r"\\{", "{", rendered)
     rendered = re.sub(r"\\}", "}", rendered)
 
-    # find any {words} that weren't replaced
-    unmatched = re.findall(r"{\w+}", rendered)
-
-    return (rendered, unmatched)
+    return rendered, unmatched
 
 
 class DateTimeFormatter:
@@ -163,6 +277,12 @@ class DateTimeFormatter:
     def __init__(self, dt: datetime.datetime):
         self.dt = dt
 
+    @property
+    def date(self):
+        """ ISO date in form 2020-03-22 """
+        date = self.dt.date().isoformat()
+        return date
+
     @property
     def year(self):
         """ 4 digit year """

osxphotos/utils.py

@@ -1,12 +1,13 @@
 import glob
 import logging
 import os.path
+import pathlib
 import platform
 import sqlite3
 import subprocess
+import sys
 import tempfile
 import urllib.parse
-import pathlib
 from plistlib import load as plistload
 
 import CoreFoundation

tests/Test-10.14.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-03-19T20:25:48Z</date>
+	<date>2020-03-27T04:00:09Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-03-19T22:36:41Z</date>
+	<date>2020-03-27T04:00:10Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2020-03-15T20:18:33Z</date>
+	<date>2020-03-27T03:59:54Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/recovery/Info.plist

@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-27T13:16:43Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2020-03-19T20:27:27Z</date>
+	<date>2020-03-27T04:02:59Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>

tests/Test-10.15.1.photoslibrary/database/Photos.sqlite.lock

@@ -7,7 +7,7 @@
 	<key>hostuuid</key>
 	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
 	<key>pid</key>
-	<integer>441</integer>
+	<integer>436</integer>
 	<key>processname</key>
 	<string>photolibraryd</string>
 	<key>uid</key>

tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist

@@ -3,24 +3,24 @@
 <plist version="1.0">
 <dict>
 	<key>BackgroundHighlightCollection</key>
-	<date>2020-03-22T12:51:16Z</date>
+	<date>2020-03-28T16:32:26Z</date>
 	<key>BackgroundHighlightEnrichment</key>
-	<date>2020-03-22T12:51:16Z</date>
+	<date>2020-03-28T16:32:25Z</date>
 	<key>BackgroundJobAssetRevGeocode</key>
-	<date>2020-03-22T12:51:17Z</date>
+	<date>2020-03-28T16:32:26Z</date>
 	<key>BackgroundJobSearch</key>
-	<date>2020-03-22T12:51:17Z</date>
+	<date>2020-03-28T16:32:26Z</date>
 	<key>BackgroundPeopleSuggestion</key>
-	<date>2020-03-22T12:51:16Z</date>
+	<date>2020-03-28T16:32:25Z</date>
 	<key>BackgroundUserBehaviorProcessor</key>
-	<date>2020-03-22T07:13:26Z</date>
+	<date>2020-03-28T07:30:06Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundGraphConsistencyUpdateJobDateKey</key>
-	<date>2020-03-22T12:51:17Z</date>
+	<date>2020-03-28T16:35:58Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-03-22T07:13:26Z</date>
+	<date>2020-03-28T07:30:06Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-03-22T12:51:17Z</date>
+	<date>2020-03-28T16:32:26Z</date>
 	<key>SiriPortraitDonation</key>
-	<date>2020-03-22T07:13:26Z</date>
+	<date>2020-03-28T07:30:06Z</date>
 </dict>
 </plist>

tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>FaceIDModelLastGenerationKey</key>
-	<date>2020-03-22T07:13:27Z</date>
+	<date>2020-03-28T07:30:06Z</date>
 	<key>LastContactClassificationKey</key>
-	<date>2020-03-22T07:13:29Z</date>
+	<date>2020-03-28T07:30:08Z</date>
 </dict>
 </plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/database/DataModelVersion.plist

@@ -0,0 +1,18 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>DatabaseMinorVersion</key>
+	<integer>1</integer>
+	<key>DatabaseVersion</key>
+	<integer>112</integer>
+	<key>LastOpenMode</key>
+	<integer>2</integer>
+	<key>LibrarySchemaVersion</key>
+	<integer>3301</integer>
+	<key>MetaSchemaVersion</key>
+	<integer>2</integer>
+	<key>createDate</key>
+	<date>2020-03-27T13:51:11Z</date>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -0,0 +1,25 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>Photos</key>
+	<dict>
+		<key>CollapsedSidebarSectionIdentifiers</key>
+		<array/>
+		<key>ExpandedSidebarItemIdentifiers</key>
+		<array>
+			<string>TopLevelAlbums</string>
+			<string>TopLevelSlideshows</string>
+		</array>
+		<key>lastKnownItemCounts</key>
+		<dict>
+			<key>other</key>
+			<integer>0</integer>
+			<key>photos</key>
+			<integer>6</integer>
+			<key>videos</key>
+			<integer>0</integer>
+		</dict>
+	</dict>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
+	<date>2020-03-27T13:51:13Z</date>
+	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
+	<date>2020-03-27T13:51:13Z</date>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/private/com.apple.photoanalysisd/VisionService/PersonPromoter

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>ProcessedInQuiescentState</key>
+	<true/>
+	<key>Version</key>
+	<integer>3</integer>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/private/com.apple.photoanalysisd/VisionService/clustererState.plist

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>PVClustererBringUpState</key>
+	<integer>50</integer>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/private/com.apple.photoanalysisd/VisionService/faceWorkerState.plist

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>IncrementalPersonProcessingStage</key>
+	<integer>4</integer>
+	<key>PersonBuilderLastMinimumFaceGroupSizeForCreatingMergeCandidates</key>
+	<integer>15</integer>
+	<key>PersonBuilderMergeCandidatesEnabled</key>
+	<true/>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -0,0 +1,16 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>PLLanguageAndLocaleKey</key>
+	<string>en-US:en_US</string>
+	<key>PLLastGeoProviderIdKey</key>
+	<string>7618</string>
+	<key>PLLastLocationInfoFormatVer</key>
+	<integer>12</integer>
+	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
+	<integer>1</integer>
+	<key>PLLastRevGeoVerFileFetchDateKey</key>
+	<date>2020-03-27T13:51:12Z</date>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>LastHistoryRowId</key>
+	<integer>171</integer>
+	<key>LibraryBuildTag</key>
+	<string>9689BC67-F0CE-460F-B268-CF82E1F1BFC5</string>
+	<key>LibrarySchemaVersion</key>
+	<integer>3301</integer>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/resources/moments/serverinfo.plist

@@ -0,0 +1,47 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>FileVersion</key>
+	<integer>11</integer>
+	<key>Source</key>
+	<dict>
+		<key>35230</key>
+		<dict>
+			<key>CountryMinVersions</key>
+			<dict>
+				<key>OTHER</key>
+				<integer>1</integer>
+			</dict>
+			<key>CurrentVersion</key>
+			<integer>1</integer>
+			<key>NoResultErrorIsSuccess</key>
+			<true/>
+		</dict>
+		<key>57879</key>
+		<dict>
+			<key>CountryMinVersions</key>
+			<dict>
+				<key>OTHER</key>
+				<integer>1</integer>
+			</dict>
+			<key>CurrentVersion</key>
+			<integer>1</integer>
+			<key>NoResultErrorIsSuccess</key>
+			<true/>
+		</dict>
+		<key>7618</key>
+		<dict>
+			<key>AddCountyIfNeeded</key>
+			<true/>
+			<key>CountryMinVersions</key>
+			<dict>
+				<key>OTHER</key>
+				<integer>10</integer>
+			</dict>
+			<key>CurrentVersion</key>
+			<integer>10</integer>
+		</dict>
+	</dict>
+</dict>
+</plist>

tests/Test-Places-High-Sierra-10.13.6.photoslibrary/resources/recovery/Info.plist

@@ -0,0 +1,31 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>DatabaseMinorVersion</key>
+	<integer>1</integer>
+	<key>DatabaseVersion</key>
+	<integer>112</integer>
+	<key>HistoricalMarker</key>
+	<dict>
+		<key>LastHistoryRowId</key>
+		<integer>171</integer>
+		<key>LibraryBuildTag</key>
+		<string>9689BC67-F0CE-460F-B268-CF82E1F1BFC5</string>
+		<key>LibrarySchemaVersion</key>
+		<integer>3301</integer>
+	</dict>
+	<key>LibrarySchemaVersion</key>
+	<integer>3301</integer>
+	<key>MetaSchemaVersion</key>
+	<integer>2</integer>
+	<key>SnapshotComplete</key>
+	<true/>
+	<key>SnapshotCompletedDate</key>
+	<date>2020-03-27T13:51:11Z</date>
+	<key>SnapshotLastValidated</key>
+	<date>2020-03-27T13:51:11Z</date>
+	<key>SnapshotTables</key>
+	<dict/>
+</dict>
+</plist>