Version bump for bug fix

Update exiftool.py to preserve file modification time, thanks to @hhoeck

.github/workflows/pythonpackage.yml

@@ -1,6 +1,6 @@
-name: Python package
+name: Tests
 
-on: [push]
+on: [push, pull_request]
 
 jobs:
   build:
@@ -9,7 +9,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [3.8]
+        python-version: [3.7, 3.8]
 
     steps:
     - uses: actions/checkout@v1

CHANGELOG.md

@@ -4,6 +4,180 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.34.2](https://github.com/RhetTbull/osxphotos/compare/v0.34.1...v0.34.2)
+
+> 14 September 2020
+
+- Partial fix for issue #213 [`459d91d`](https://github.com/RhetTbull/osxphotos/commit/459d91d7b11dbd4b0564906c1689b60dc5b64642)
+
+#### [v0.34.1](https://github.com/RhetTbull/osxphotos/compare/v0.34.0...v0.34.1)
+
+> 13 September 2020
+
+- Fixed exception handling in export [`eb00ffd`](https://github.com/RhetTbull/osxphotos/commit/eb00ffd73737ef4832229e4e6fd8dc4ccb0b8539)
+- Updated README.md [`a1776fa`](https://github.com/RhetTbull/osxphotos/commit/a1776fa14850275ad6b02ece80bbe8ce908fa836)
+
+#### [v0.34.0](https://github.com/RhetTbull/osxphotos/compare/v0.33.8...v0.34.0)
+
+> 7 September 2020
+
+- Added --skip-original-if-edited for issue #159 [`5f2d401`](https://github.com/RhetTbull/osxphotos/commit/5f2d401048850fd68f31b37a7e71abc11ca80dc5)
+- Still working on issue #208 [`58b3869`](https://github.com/RhetTbull/osxphotos/commit/58b3869a7cce7cb3f211599e544d7e5426ceb4a6)
+
+#### [v0.33.8](https://github.com/RhetTbull/osxphotos/compare/v0.33.7...v0.33.8)
+
+> 31 August 2020
+
+- Fixed sidecar collisions, closes #210 [`#210`](https://github.com/RhetTbull/osxphotos/issues/210)
+
+#### [v0.33.7](https://github.com/RhetTbull/osxphotos/compare/v0.33.5...v0.33.7)
+
+> 31 August 2020
+
+- typo fix - thanks to @dmd  [`#212`](https://github.com/RhetTbull/osxphotos/pull/212)
+- Normalize unicode for issue #208 [`a36eb41`](https://github.com/RhetTbull/osxphotos/commit/a36eb416b19284477922b6a5f837f4040327138b)
+- Added force_download.py to examples [`b611d34`](https://github.com/RhetTbull/osxphotos/commit/b611d34d19db480af72f57ef55eacd0a32c8d1e8)
+- Added photoshop:SidecarForExtension to XMP, partial fix for #210 [`60d96a8`](https://github.com/RhetTbull/osxphotos/commit/60d96a8f563882fba2365a6ab58c1276725eedaa)
+- Updated README.md [`c9b1518`](https://github.com/RhetTbull/osxphotos/commit/c9b15186a022d91248451279e5f973e3f2dca4b4)
+- Update README.md [`42e8fba`](https://github.com/RhetTbull/osxphotos/commit/42e8fba125a3c6b1bd0d538f2af511aabfbeb478)
+
+#### [v0.33.5](https://github.com/RhetTbull/osxphotos/compare/v0.33.3...v0.33.5)
+
+> 25 August 2020
+
+- Fixed DST handling for from_date/to_date, closes #193 (again) [`#193`](https://github.com/RhetTbull/osxphotos/issues/193)
+- Added raw timestamps to PhotoInfo._info [`0f457a4`](https://github.com/RhetTbull/osxphotos/commit/0f457a4082a4eebc42a5df2160a02ad987b6f96c)
+
+#### [v0.33.3](https://github.com/RhetTbull/osxphotos/compare/v0.33.2...v0.33.3)
+
+> 23 August 2020
+
+- Fixed portrait for Catalina/Big Sur; see issue #203 [`1f717b0`](https://github.com/RhetTbull/osxphotos/commit/1f717b05794c2088c7c15d2aab0c5d24b6309c06)
+
+#### [v0.33.2](https://github.com/RhetTbull/osxphotos/compare/v0.33.0...v0.33.2)
+
+> 23 August 2020
+
+- Closes issue #206, adds --touch-file [`#207`](https://github.com/RhetTbull/osxphotos/pull/207)
+- Touch files - fixes #194 -- thanks to @PabloKohan  [`#205`](https://github.com/RhetTbull/osxphotos/pull/205)
+- Refactor/cleanup _export_photo - thanks to @PabloKohan  [`#204`](https://github.com/RhetTbull/osxphotos/pull/204)
+- Finished --touch-file, closes #206 [`#206`](https://github.com/RhetTbull/osxphotos/issues/206)
+- Merge pull request #205 from PabloKohan/touch_files__fix_194 [`#194`](https://github.com/RhetTbull/osxphotos/issues/194)
+- --touch-file now working with --update [`6c11e3f`](https://github.com/RhetTbull/osxphotos/commit/6c11e3fa5b5b05b98b9fdbb0e59e3a78c7dff980)
+- Refactor/cleanup _export_photo [`eefa1f1`](https://github.com/RhetTbull/osxphotos/commit/eefa1f181f4fd7b027ae69abd2b764afb590c081)
+- Fixed touch tests [`1bf7105`](https://github.com/RhetTbull/osxphotos/commit/1bf7105737fbd756064a2f9ef4d4bbd0b067978c)
+- Working on issue 206 [`ebd878a`](https://github.com/RhetTbull/osxphotos/commit/ebd878a075983ef3df0b1ead1a725e01508721f8)
+- Working on issue #206 [`c9c9202`](https://github.com/RhetTbull/osxphotos/commit/c9c920220545dc27c8cb1379d7bde15987cce72c)
+
+#### [v0.33.0](https://github.com/RhetTbull/osxphotos/compare/v0.32.0...v0.33.0)
+
+> 17 August 2020
+
+- Replaced call to which, closes #171 [`#171`](https://github.com/RhetTbull/osxphotos/issues/171)
+- Added contributors to README.md, closes #200 [`#200`](https://github.com/RhetTbull/osxphotos/issues/200)
+- Added tests for 10.15.6 [`d2deeff`](https://github.com/RhetTbull/osxphotos/commit/d2deefff834e46e1a26adc01b1b025ac839dbc78)
+- Added ImportInfo for Photos 5+ [`98e4170`](https://github.com/RhetTbull/osxphotos/commit/98e417023ec5bd8292b25040d0844f3706645950)
+- Update README.md [`360c8d8`](https://github.com/RhetTbull/osxphotos/commit/360c8d8e1b4760e95a8b71b3a0bf0df4fb5adaf5)
+- Update README.md [`868cda8`](https://github.com/RhetTbull/osxphotos/commit/868cda8482ce6b29dd00e04a209d40550e6b128b)
+
+#### [v0.32.0](https://github.com/RhetTbull/osxphotos/compare/v0.31.2...v0.32.0)
+
+> 9 August 2020
+
+- Alpha support for MacOS Big Sur/10.16, see issue #187 [`6acf9ac`](https://github.com/RhetTbull/osxphotos/commit/6acf9acd6364e1996158179493d128ec0958e652)
+
+#### [v0.31.2](https://github.com/RhetTbull/osxphotos/compare/v0.31.0...v0.31.2)
+
+> 9 August 2020
+
+- Fixed from_date and to_date to be timezone aware, closes #193 [`#193`](https://github.com/RhetTbull/osxphotos/issues/193)
+- Added test for valid XMP file, closes #197 [`#197`](https://github.com/RhetTbull/osxphotos/issues/197)
+- Dropped py36 due to datetime.fromisoformat [`a714ae0`](https://github.com/RhetTbull/osxphotos/commit/a714ae0af089b13acf70c4f29934393aa48ed222)
+- Added --uuid-from-file to CLI [`840e993`](https://github.com/RhetTbull/osxphotos/commit/840e9937bede407ef55972a361618683245e086b)
+- Added write_uuid_to_file.applescript to utils [`bea770b`](https://github.com/RhetTbull/osxphotos/commit/bea770b322d21cf3f8245d20e182006247cb71d6)
+- Updated README.md [`002fce8`](https://github.com/RhetTbull/osxphotos/commit/002fce8e93edd936d4b866118ae6d4c94e5d6744)
+- Added py37 [`d0ec862`](https://github.com/RhetTbull/osxphotos/commit/d0ec8620c721fe7576ab7d519a5eaac4d17a317e)
+
+#### [v0.31.0](https://github.com/RhetTbull/osxphotos/compare/v0.30.13...v0.31.0)
+
+> 27 July 2020
+
+- Initial FaceInfo support for Issue #21 [`6f29cda`](https://github.com/RhetTbull/osxphotos/commit/6f29cda99f1b8d94a95597c7046620cf21fecae4)
+- Updated Github Actions to run on PR [`9fc4f76`](https://github.com/RhetTbull/osxphotos/commit/9fc4f762193699dd45b586b51aa2d3066928aab1)
+
+#### [v0.30.13](https://github.com/RhetTbull/osxphotos/compare/v0.30.12...v0.30.13)
+
+> 23 July 2020
+
+- This reverts commit b7f4b739de978991def8ae2dca0f4e4b2881f56d, reversing [`#191`](https://github.com/RhetTbull/osxphotos/pull/191)
+- Fix findfiles not to fail on missing/invalid dir [`#192`](https://github.com/RhetTbull/osxphotos/pull/192)
+- Revert "Fix FileExistsError when filename differs only in case and export-as-hardlink (Bug#133)" [`#191`](https://github.com/RhetTbull/osxphotos/pull/191)
+- Fix FileExistsError when filename differs only in case and export-as-hardlink (Bug#133) [`#190`](https://github.com/RhetTbull/osxphotos/pull/190)
+- Fix findfiles not to fail on missing/invalid dir [`8c10b61`](https://github.com/RhetTbull/osxphotos/commit/8c10b61e90abbcfdff472bad4bb760558c7b850c)
+- Revert "Fix FileExistsError when filename differs only in case and export-as-hardlink (Bug#133)" [`f8e62d8`](https://github.com/RhetTbull/osxphotos/commit/f8e62d8f5ed26814f02383426237fd4c99a7ad04)
+- Fix FileExistsError when filename differs only in case and export-as-hardlink [`d52b387`](https://github.com/RhetTbull/osxphotos/commit/d52b387a294e68ebf0580a202ea70b97205560ef)
+- Version bump for bug fix [`cf4dca1`](https://github.com/RhetTbull/osxphotos/commit/cf4dca10c02d5f3f6132ab1572a698379b667e48)
+
+#### [v0.30.12](https://github.com/RhetTbull/osxphotos/compare/v0.30.10...v0.30.12)
+
+> 18 July 2020
+
+- Implemented PersonInfo, closes #181 [`#181`](https://github.com/RhetTbull/osxphotos/issues/181)
+- Updated dependencies, now supports py36, py37, py38 [`6688d1f`](https://github.com/RhetTbull/osxphotos/commit/6688d1ff6491f2e7e155946b265ef8b5d8929441)
+- Update README.md [`3526881`](https://github.com/RhetTbull/osxphotos/commit/3526881ec872cc009b0d8936f366afcfff166d42)
+
+#### [v0.30.10](https://github.com/RhetTbull/osxphotos/compare/v0.30.9...v0.30.10)
+
+> 6 July 2020
+
+- Bug fix for empty albums [`1ef518c`](https://github.com/RhetTbull/osxphotos/commit/1ef518cc3e9efbe9d4c16aa3d36c6dc6db86798e)
+
+#### [v0.30.9](https://github.com/RhetTbull/osxphotos/compare/v0.30.7...v0.30.9)
+
+> 6 July 2020
+
+- Refactored person processing to enable implementation of #181 [`fcff8ec`](https://github.com/RhetTbull/osxphotos/commit/fcff8ec5f8286b28e7d8559b40b5808a7b59cc15)
+- AlbumInfo.photos now returns photos in album sort order [`9d820a0`](https://github.com/RhetTbull/osxphotos/commit/9d820a0557944340d0c664a6c3497d138c6100d5)
+
+#### [v0.30.7](https://github.com/RhetTbull/osxphotos/compare/v0.30.6...v0.30.7)
+
+> 4 July 2020
+
+- Bug fix for keywords, persons in deleted photos [`df75a05`](https://github.com/RhetTbull/osxphotos/commit/df75a05645a88b31daa411f960d99ade71efc908)
+
+#### [v0.30.6](https://github.com/RhetTbull/osxphotos/compare/v0.30.5...v0.30.6)
+
+> 3 July 2020
+
+- Added height, width, orientation, filesize to json, str) [`8c3af0a`](https://github.com/RhetTbull/osxphotos/commit/8c3af0a4e4e49d9bbb33e809973d958334e44dca)
+
+#### [v0.30.5](https://github.com/RhetTbull/osxphotos/compare/v0.30.4...v0.30.5)
+
+> 3 July 2020
+
+- Added height, width, orientation, filesize, closes #163 [`#163`](https://github.com/RhetTbull/osxphotos/issues/163)
+
+#### [v0.30.4](https://github.com/RhetTbull/osxphotos/compare/v0.30.3...v0.30.4)
+
+> 3 July 2020
+
+- Added GPS location to XMP sidecar, closes #175 [`#175`](https://github.com/RhetTbull/osxphotos/issues/175)
+- Updated README.md [`7806e05`](https://github.com/RhetTbull/osxphotos/commit/7806e05673775ded231e65f53f3a1d5095a4b4e1)
+
+#### [v0.30.3](https://github.com/RhetTbull/osxphotos/compare/v0.30.2...v0.30.3)
+
+> 29 June 2020
+
+- Added --description-template to CLI, closes #166 [`#166`](https://github.com/RhetTbull/osxphotos/issues/166)
+- Added expand_inplace to PhotoTemplate.render [`ff03287`](https://github.com/RhetTbull/osxphotos/commit/ff0328785f3ea14b1c8ae2b7d1a9b07e8aef0777)
+- Updated README.md [`5950707`](https://github.com/RhetTbull/osxphotos/commit/59507077bafe39a17bc23babe6d6c52e1f502a53)
+
+#### [v0.30.2](https://github.com/RhetTbull/osxphotos/compare/v0.30.1...v0.30.2)
+
+> 28 June 2020
+
+- Added --deleted, --deleted-only to CLI, closes #179 [`#179`](https://github.com/RhetTbull/osxphotos/issues/179)
+
 #### [v0.30.1](https://github.com/RhetTbull/osxphotos/compare/v0.30.0...v0.30.1)
 
 > 27 June 2020

README.md

@@ -15,9 +15,12 @@
     + [PhotoInfo](#photoinfo)
     + [ExifInfo](#exifinfo)
     + [AlbumInfo](#albuminfo)
+    + [ImportInfo](#importinfo)
     + [FolderInfo](#folderinfo)
     + [PlaceInfo](#placeinfo)
     + [ScoreInfo](#scoreinfo)
+    + [PersonInfo](#personinfo)
+    + [FaceInfo](#faceinfo)
     + [Template Substitutions](#template-substitutions)
     + [Utility Functions](#utility-functions)
   * [Examples](#examples)
@@ -35,9 +38,11 @@ OSXPhotos provides the ability to interact with and query Apple's Photos.app lib
 
 ## Supported operating systems
 
-Only works on MacOS (aka Mac OS X). Tested on MacOS 10.12.6 / Photos 2.0, 10.13.6 / Photos 3.0, MacOS 10.14.5, 10.14.6 / Photos 4.0, MacOS 10.15.1 - 10.15.5 / Photos 5.0. 
+Only works on MacOS (aka Mac OS X). Tested on MacOS 10.12.6 / Photos 2.0, 10.13.6 / Photos 3.0, MacOS 10.14.5, 10.14.6 / Photos 4.0, MacOS 10.15.1 - 10.15.6 / Photos 5.0.
 
-Requires python >= 3.8. You can probably get this to run with Python 3.6 or 3.7 (see notes [below](#Installation-instructions)) but only 3.8+ is officially supported. 
+Alpha support for MacOS 10.16/MacOS 11 Big Sur Beta / Photos 6.0.
+
+Requires python >= 3.7. 
 
 This package will read Photos databases for any supported version on any supported OS version.  E.g. you can read a database created with Photos 4.0 on MacOS 10.14 on a machine running MacOS 10.12.
 
@@ -48,14 +53,12 @@ OSXPhotos uses setuptools, thus simply run:
 
 	python3 setup.py install
 
-If you're using python 3.6 or 3.7, you'll need to do this first to get around an issue with bpylist2:
-
-	pip install -r requirements.txt
-
-You can also install directly from [pypi](https://pypi.org/) but you must use python >= 3.8 to avoid an error with bpylist2.  The package currently works fine with python 3.6 or 3.7 but I know of no way to get `pip` to install the right dependencies. 
+You can also install directly from [pypi](https://pypi.org/project/osxphotos/):
 
     pip install osxphotos
 	
+**WARNING** The git repo for this project is very large (> 1GB) because it contains multiple Photos libraries used for testing on different versions of MacOS.  If you just want to use the osxphotos package in your own code, I recommend you install the latest version from [PyPI](https://pypi.org/project/osxphotos/). If you just want to use the command line utility, you can download a pre-built executable of the latest [release](https://github.com/RhetTbull/osxphotos/releases) or you can install via `pip` which also installs the command line app.  If you aren't comfortable with running python on your Mac, start with the pre-built executable.
+
 ## Command Line Usage
 
 This package will install a command line utility called `osxphotos` that allows you to query the Photos database.  Alternatively, you can also run the command line utility like this: `python3 -m osxphotos`
@@ -140,6 +143,9 @@ Options:
                                   searches top level folders (e.g. does not
                                   look at subfolders)
   --uuid UUID                     Search for photos with UUID(s).
+  --uuid-from-file FILE           Search for photos with UUID(s) loaded from
+                                  FILE. Format is a single UUID per line.
+                                  Lines preceeded with # are ignored.
   --title TITLE                   Search for TITLE in title of photo.
   --no-title                      Search for photos with no title.
   --description DESC              Search for DESC in description of photo.
@@ -199,14 +205,14 @@ Options:
                                   both images and movies).
   --only-photos                   Search only for photos/images (default
                                   searches both images and movies).
-  --from-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
-                                  Search by start item date, e.g.
-                                  2000-01-12T12:00:00 or 2000-12-31 (ISO 8601
-                                  w/o TZ).
-  --to-date [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
-                                  Search by end item date, e.g.
-                                  2000-01-12T12:00:00 or 2000-12-31 (ISO 8601
-                                  w/o TZ).
+  --from-date DATETIME            Search by start item date, e.g.
+                                  2000-01-12T12:00:00,
+                                  2001-01-12T12:00:00-07:00, or 2000-12-31
+                                  (ISO 8601).
+  --to-date DATETIME              Search by end item date, e.g.
+                                  2000-01-12T12:00:00,
+                                  2001-01-12T12:00:00-07:00, or 2000-12-31
+                                  (ISO 8601).
   --deleted                       Include photos from the 'Recently Deleted'
                                   folder.
   --deleted-only                  Include only photos from the 'Recently
@@ -218,6 +224,8 @@ Options:
   --export-as-hardlink            Hardlink files instead of copying them.
                                   Cannot be used with --exiftool which creates
                                   copies of the files with embedded EXIF data.
+  --touch-file                    Sets the file's modification time to match
+                                  photo date.
   --overwrite                     Overwrite existing files. Default behavior
                                   is to add (1), (2), etc to filename if file
                                   already exists. Use this with caution as it
@@ -228,6 +236,8 @@ Options:
                                   DEST/2019/12/20/photoname.jpg).
   --skip-edited                   Do not export edited version of photo if an
                                   edited version exists.
+  --skip-original-if-edited       Do not export original if there is an edited
+                                  version (exports only the edited version).
   --skip-bursts                   Do not export all associated burst images in
                                   the library if a photo is a burst photo.
   --skip-live                     Do not export the associated live video
@@ -253,6 +263,16 @@ Options:
                                   --keyword-template "{folder_album}"
                                   --keyword-template "{created.year}" See
                                   Templating System below.
+  --description-template TEMPLATE
+                                  For use with --exiftool, --sidecar; specify
+                                  a template string to use as description in
+                                  the form '{name,DEFAULT}' This is the same
+                                  format as --directory.  For example, if you
+                                  wanted to append 'exported with osxphotos on
+                                  [today's date]' to the description, you
+                                  could specify --description-template
+                                  "{descr} exported with osxphotos on
+                                  {today.date}" See Templating System below.
   --current-name                  Use photo's current filename instead of
                                   original filename for export.  Note:
                                   Starting with Photos 5, all photos are
@@ -657,7 +677,7 @@ if __name__ == "__main__":
 #### Read a Photos library database
 
 ```python
-osxphotos.PhotosDB() # not recommended, see Note below
+osxphotos.PhotosDB()
 osxphotos.PhotosDB(path)
 osxphotos.PhotosDB(dbfile=path)
 ```
@@ -668,7 +688,7 @@ Pass the path to a Photos library or to a specific database file (e.g. "/Users/s
 
 If an invalid path is passed, PhotosDB will raise `FileNotFoundError` exception.
 
-**Note**: If neither path or dbfile is passed, PhotosDB will use get_last_library_path to open the last opened Photos library.  This usually works but is not 100% reliable.  It can also lead to loading a different library than expected if the user has held down *option* key when opening Photos to switch libraries.  It is therefore recommended you explicitely pass the path to `PhotosDB()`.  
+**Note**: If neither path or dbfile is passed, PhotosDB will use get_last_library_path to open the last opened Photos library.  This usually works but is not 100% reliable.  It can also lead to loading a different library than expected if the user has held down *option* key when opening Photos to switch libraries.  You may therefore want to explicitely pass the path to `PhotosDB()`. 
 
 #### Open the default (last opened) Photos library
 
@@ -754,6 +774,10 @@ Returns list of shared album names found in photos database (e.g. albums shared
 
 **Note**: *Only valid for Photos 5 / MacOS 10.15*; on Photos <= 4, prints warning and returns empty list.
 
+#### `import_info`
+
+Returns a list of [ImportInfo](#importinfo) objects representing the import sessions for the database.
+
 #### `folder_info`
 ```python
 # assumes photosdb is a PhotosDB object (see above)
@@ -780,7 +804,15 @@ Returns a list names of top level folder names in the database.
 persons = photosdb.persons
 ```
 
-Returns a list of the persons (faces) found in the Photos library
+Returns a list of the person names (faces) found in the Photos library.  **Note**: It is of course possible to have more than one person with the same name, e.g. "Maria Smith", in the database.  `persons` assumes these are the same person and will list only one person named "Maria Smith".  If you need more information about persons in the database, see [person_info](#dbpersoninfo).
+
+#### <a name="dbpersoninfo">`person_info`</a>
+```python
+# assumes photosdb is a PhotosDB object (see above)
+person_info = photosdb.person_info
+```
+
+Returns a list of [PersonInfo](#personinfo) objects representing persons who appear in photos in the database. 
 
 #### `keywords_as_dict`
 ```python
@@ -796,7 +828,8 @@ Returns a dictionary of keywords found in the Photos library where key is the ke
 persons_dict = photosdb.persons_as_dict
 ```
 
-Returns a dictionary of persons (faces) found in the Photos library where key is the person name and value is the count of how many times that person appears in the library (ie. how many photos are tagged with the person).  Resulting dictionary is in reverse sorted order (e.g. person who appears in the most photos is listed first).
+Returns a dictionary of persons (faces) found in the Photos library where key is the person name and value is the count of how many times that person appears in the library (ie. how many photos are tagged with the person).  Resulting dictionary is in reverse sorted order (e.g. person who appears in the most photos is listed first). **Note**: It is of course possible to have more than one person with the same name, e.g. "Maria Smith", in the database.  `persons_as_dict` assumes these are the same person and will list only one person named "Maria Smith".  If you need more information about persons in the database, see [person_info](#dbpersoninfo).
+
 
 #### `albums_as_dict`
 ```python
@@ -882,8 +915,7 @@ for row in results:
 
 conn.close()
 ```
-
-#### ` photos(keywords=None, uuid=None, persons=None, albums=None, images=True, movies=True, from_date=None, to_date=None, intrash=False)`
+#### <A name="photos">`photos(keywords=None, uuid=None, persons=None, albums=None, images=True, movies=True, from_date=None, to_date=None, intrash=False)`</a>
 
 ```python
 # assumes photosdb is a PhotosDB object (see above)
@@ -919,6 +951,8 @@ photos = photosdb.photos(
 - ```to_date```: datetime.datetime; if provided, finds photos where creation date <= to_date; default is None
 - ```intrash```: if True, finds only photos in the "Recently Deleted" or trash folder, if False does not find any photos in the trash; default is False
 
+See also [get_photo()](#getphoto) which is much faster for retrieving a single photo.
+
 If more than one of (keywords, uuid, persons, albums,from_date, to_date) is provided, they are treated as "and" criteria. E.g.
 
 Finds all photos with (keyword = "wedding" or "birthday") and (persons = "Juan Rodriguez")
@@ -993,6 +1027,9 @@ For example, in my library, Photos says I have 19,386 photos and 474 movies.  Ho
 >>>
 ```
 
+#### <a name="getphoto">`get_photo(uuid)`</A>
+Returns a single PhotoInfo instance for photo with UUID matching `uuid` or None if no photo is found matching `uuid`.  If you know the UUID of a photo, `get_photo()` is much faster than `photos`.  See also [photos()](#photos).
+
 
 ### PhotoInfo 
 PhotosDB.photos() returns a list of PhotoInfo objects.  Each PhotoInfo object represents a single photo in the Photos library.
@@ -1027,9 +1064,18 @@ Returns a list of albums the photo is contained in. See also [album_info](#album
 #### `album_info`
 Returns a list of [AlbumInfo](#AlbumInfo) objects representing the albums the photo is contained in.  See also [albums](#albums).
 
+#### `import_info`
+Returns an [ImportInfo](#importinfo) object representing the import session associated with the photo or `None` if there is no associated import session.
+
 #### `persons`
 Returns a list of the names of the persons in the photo
 
+#### <a name="photopersoninfo">`person_info`</a>
+Returns a list of [PersonInfo](#personinfo) objects representing persons in the photo.  Each PersonInfo object is associated with one or more FaceInfo objects.
+
+#### <a name="photofaceinfo">`face_info`</a>
+Returns a list of [FaceInfo](#faceinfo) objects representing faces in the photo.  Each face is associated with the a PersonInfo object.
+
 #### `path`
 Returns the absolute path to the photo on disk as a string.  **Note**: this returns the path to the *original* unedited file (see [hasadjustments](#hasadjustments)).  If the file is missing on disk, path=`None` (see [ismissing](#ismissing)).
 
@@ -1038,6 +1084,27 @@ Returns the absolute path to the edited photo on disk as a string.  If the photo
 
 **Note**: will also return None if the edited photo is missing on disk. 
 
+#### `height`
+Returns height of the photo in pixels.  If image has been edited, returns height of the edited image, otherwise returns height of the original image.  See also [original_height](#original_height).
+
+#### `width`
+Returns width of the photo in pixels.  If image has been edited, returns width of the edited image, otherwise returns width of the original image.  See also [original_width](#original_width).
+
+#### `orientation`
+Returns EXIF orientation value of the photo as integer.  If image has been edited, returns orientation of the edited image, otherwise returns orientation of the original image. See also [original_orientation](#original_orientation).
+
+#### `original_height`
+Returns height of the original photo in pixels. See also [height](#height).
+
+#### `original_width`
+Returns width of the original photo in pixels. See also [width](#width). 
+
+#### `original_orientation`
+Returns EXIF orientation value of the original photo as integer. See also [orientation](#orientation).
+
+#### `original_filesize`
+Returns size of the original photo in bytes as integer.
+
 #### `ismissing`
 Returns `True` if the original image file is missing on disk, otherwise `False`.  This can occur if the file has been uploaded to iCloud but not yet downloaded to the local library or if the file was deleted or imported from a disk that has been unmounted and user hasn't enabled "Copy items to the Photos library" in Photos preferences. **Note**: this status is computed based on data in the Photos library and `ismissing` does not verify if the photo is actually missing. See also [path](#path).
 
@@ -1256,11 +1323,13 @@ If overwrite=False and increment=False, export will fail if destination file alr
 
 #### <a name="rendertemplate">`render_template()`</a>
 
-`render_template(template_str, none_str = "_", path_sep = None)`
+`render_template(template_str, none_str = "_", path_sep = None, expand_inplace = False, inplace_sep = None)`
 Render template string for photo.  none_str is used if template substitution results in None value and no default specified. 
 - `template_str`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None.  This is useful for values which are not always present, for example reverse geolocation data.
 - `none_str`: optional str to use as substitution when template value is None and no default specified in the template string.  default is "_".
 - `path_sep`: optional character to use as path separator, default is os.path.sep
+- `expand_inplace`: expand multi-valued substitutions in-place as a single string instead of returning individual strings
+- `inplace_sep`: optional string to use as separator between multi-valued keywords with expand_inplace; default is ','
 
 Returns a tuple of (rendered, unmatched) where rendered is a list of rendered strings with all substitutions made and unmatched is a list of any strings that resembled a template substitution but did not match a known substitution. E.g. if template contained "{foo}", unmatched would be ["foo"].
 
@@ -1320,8 +1389,17 @@ Returns the universally unique identifier (uuid) of the album.  This is how Phot
 #### `title`
 Returns the title or name of the album.
 
-#### `photos`
-Returns a list of [PhotoInfo](#PhotoInfo) objects representing each photo contained in the album.
+#### <a name="albumphotos">`photos`</a>
+Returns a list of [PhotoInfo](#PhotoInfo) objects representing each photo contained in the album sorted in the same order as in Photos. (e.g. if photos were manually sorted in the Photos albums, photos returned by `photos` will be in same order as they appear in the Photos album)
+
+#### `creation_date`
+Returns the creation date as a timezone aware datetime.datetime object of the album.
+
+#### `start_date`
+Returns the date of earliest photo in the album as a timezone aware datetime.datetime object.
+
+#### `end_date`
+Returns the date of latest photo in the album as a timezone aware datetime.datetime object.
 
 #### `folder_list`
 Returns a hierarchical list of [FolderInfo](#FolderInfo) objects representing the folders the album is contained in.  For example, if album "AlbumInFolder" is in SubFolder2 of Folder1 as illustrated below, would return a list of `FolderInfo` objects representing ["Folder1", "SubFolder2"] 
@@ -1348,6 +1426,25 @@ Photos Library
 #### `parent`
 Returns a [FolderInfo](#FolderInfo) object representing the albums parent folder or `None` if album is not a in a folder.
 
+### ImportInfo 
+PhotosDB.import_info returns a list of ImportInfo objects.  Each ImportInfo object represents an import session in the library.  PhotoInfo.import_info returns a single ImportInfo object representing the import session for the photo (or `None` if no associated import session).
+
+**Note**: Photos 5+ only.  Not implemented for Photos version <= 4.
+
+#### `uuid`
+Returns the universally unique identifier (uuid) of the import session.  This is how Photos keeps track of individual objects within the database.
+
+#### <a name="importphotos">`photos`</a>
+Returns a list of [PhotoInfo](#PhotoInfo) objects representing each photo contained in the import session.
+
+#### `creation_date`
+Returns the creation date as a timezone aware datetime.datetime object of the import session.
+
+#### `start_date`
+Returns the start date as a timezone aware datetime.datetime object for when the import session bega.
+
+#### `end_date`
+Returns the end date as a timezone aware datetime.datetime object for when the import session completed.
 
 ### FolderInfo 
 PhotosDB.folder_info returns a list of FolderInfo objects representing the top level folders in the library.  Each FolderInfo object represents a single folder in the Photos library.
@@ -1495,6 +1592,118 @@ Example: find your "best" photo of food
 >>> best_food_photo = sorted([p for p in photos if "food" in p.labels_normalized], key=lambda p: p.score.overall, reverse=True)[0]
 ```
 
+### PersonInfo
+[PhotosDB.person_info](#dbpersoninfo) and [PhotoInfo.person_info](#photopersoninfo) return a list of PersonInfo objects represents persons in the database and in a photo, respectively.  The PersonInfo class has the following properties and methods.
+
+#### `name`
+Returns the full name of the person represented in the photo. For example, "Maria Smith".
+
+#### `display_name`
+Returns the display name of the person represented in the photo. For example, "Maria".
+
+#### `uuid`
+Returns the UUID of the person as stored in the Photos library database.
+
+#### `keyphoto`
+Returns a PhotoInfo instance for the photo designated as the key photo for the person. This is the Photos uses to display the person's face thumbnail in Photos' "People" view. 
+
+#### `facecount`
+Returns a count of how many times this person appears in images in the database.
+
+#### <a name="personphotos">`photos`</a>
+Returns a list of PhotoInfo objects representing all photos the person appears in.
+
+#### <a name="personfaceinfo">`face_info`</a>
+Returns a list of [FaceInfo](#faceinfo) objects associated with this person sorted by quality score. Highest quality face is result[0] and lowest quality face is result[n].
+
+#### `json()`
+Returns a json string representation of the PersonInfo instance.
+
+### FaceInfo 
+[PhotoInfo.face_info](#photofaceinfo) return a list of FaceInfo objects representing detected faces in a photo.  The FaceInfo class has the following properties and methods.
+
+#### `uuid`
+UUID of the face.
+
+#### `name`
+Full name of the person represented by the face or None if person hasn't been given a name in Photos.  This is a shortcut for `FaceInfo.person_info.name`.
+
+#### `asset_uuid`
+UUID of the photo this face is associated with.
+
+#### `person_info`
+[PersonInfo](#personinfo) object associated with this face.
+
+#### `photo`
+[PhotoInfo](#photoinfo) object representing the photo that contains this face.
+
+#### `face_rect()`
+Returns list of x, y coordinates as tuples `[(x0, y0), (x1, y1)]` representing the corners of rectangular region that contains the face.  Coordinates are in same format and [reference frame](https://pillow.readthedocs.io/en/stable/handbook/concepts.html#coordinate-system) as used by [Pillow](https://pypi.org/project/Pillow/) imaging library.  **Note**: face_rect() and all other properties/methods that return coordinates refer to the *current version* of the image. E.g. if the image has been edited ([`PhotoInfo.hasadjustments`](#hasadjustments)), these refer to [`PhotoInfo.path_edited`](#pathedited).  If the image has no adjustments, these coordinates refer to the original photo ([`PhotoInfo.path`](#path)).
+
+#### `center`
+Coordinates as (x, y) tuple for the center of the detected face.
+
+#### `mouth`
+Coordinates as (x, y) tuple for the mouth of the detected face.
+
+#### `left_eye`
+Coordinates as (x, y) tuple for the left eye of the detected face.
+
+#### `right_eye`
+Coordinates as (x, y) tuple for the right eye of the detected face.
+
+#### `size_pixels`
+Diameter of detected face region in pixels.
+
+#### `roll_pitch_yaw()`
+Roll, pitch, and yaw of face region in radians.  Returns a tuple of (roll, pitch, yaw)
+
+#### roll
+Roll of face region in radians. 
+
+#### pitch 
+Pitch of face region in radians. 
+
+#### yaw 
+Yaw of face region in radians. 
+
+#### `Additional properties`
+The following additional properties are also available but are not yet fully documented.
+
+- `center_x`: x coordinate of center of face in Photos' internal reference frame
+- `center_y`: y coordinate of center of face in Photos' internal reference frame
+- `mouth_x`: x coordinate of mouth in Photos' internal reference frame
+- `mouth_y`: y coordinate of mouth in Photos' internal reference frame
+- `left_eye_x`: x coordinate of left eye in Photos' internal reference frame
+- `left_eye_y`: y coordinate of left eye in Photos' internal reference frame
+- `right_eye_x`: x coordinate of right eye in Photos' internal reference frame
+- `right_eye_y`: y coordinate of right eye in Photos' internal reference frame
+- `size`: size of face region in Photos' internal reference frame
+- `quality`: quality measure of detected face
+- `source_width`: width in pixels of photo
+- `source_height`: height in pixels of photo
+- `has_smile`: 
+- `left_eye_closed`: 
+- `right_eye_closed`:
+- `manual`: 
+- `face_type`:
+- `age_type`:
+- `bald_type`:
+- `eye_makeup_type`:
+- `eye_state`:
+- `facial_hair_type`:
+- `gender_type`:
+- `glasses_type`:
+- `hair_color_type`:
+- `lip_makeup_type`:
+- `smile_type`:
+
+#### `asdict()`
+Returns a dictionary representation of the FaceInfo instance.
+
+#### `json()`
+Returns a JSON representation of the FaceInfo instance.
+
 ### Template Substitutions
 
 The following substitutions are availabe for use with `PhotoInfo.render_template()` 
@@ -1640,6 +1849,7 @@ if __name__ == "__main__":
 ## Related Projects
 
 - [rhettbull/photosmeta](https://github.com/rhettbull/photosmeta): uses osxphotos and [exiftool](https://exiftool.org/) to apply metadata from Photos as exif data in the photo files.  Can also export photos while preserving metadata and also apply Photos keywords as spotlight tags to make it easier to search for photos using spotlight.  This is mostly made obsolete by osxphotos.  The one feature that photosmeta has that osxphotos does not is ability to update the metadata of the actual photo files in the Photos library without exporting them. (Use with caution!)
+- [rhettbull/PhotoScript](https://github.com/RhetTbull/PhotoScript): python wrapper around Photos' applescript API allowing automation of Photos (including creation/deletion of items) from python.
 - [patrikhson/photo-export](https://github.com/patrikhson/photo-export): Exports older versions of Photos databases.  Provided the inspiration for osxphotos.
 - [orangeturtle739/photos-export](https://github.com/orangeturtle739/photos-export): Set of scripts to export Photos libraries.
 - [ndbroadbent/icloud_photos_downloader](https://github.com/ndbroadbent/icloud_photos_downloader): Download photos from iCloud.  Currently unmaintained.
@@ -1657,12 +1867,29 @@ If you have an interesting example that shows usage of this package, submit an i
 
 Testing against "real world" Photos libraries would be especially helpful.  If you discover issues in testing against your Photos libraries, please open an issue.  I've done extensive testing against my own Photos library but that's a since data point and I'm certain there are issues lurking in various edge cases I haven't discovered yet.
 
+### Contributors
+
+Thank-you to the following people who have contributed to improving osxphotos!  If I've inadvertently left you off, please open an issue or send me a note.
+
+- [britiscurious](https://github.com/britiscurious)
+- [Michel Wortmann](https://github.com/mwort)
+- [hshore29](https://github.com/hshore29)
+- [Pablo 'merKur' Kohan](https://github.com/PabloKohan)
+- [Jean-Yves Stervinou](https://github.com/jystervinou)
+- [Thibault Deutsch](https://github.com/dethi)
+- [grundsch](https://github.com/grundsch)
+- [Ag Primatic](https://github.com/agprimatic)
+- [Daniel M. Drucker](https://github.com/dmd)
+- [Horst Höck](https://github.com/hhoeck)
+
+
 ## Known Bugs
 
-My goal is make osxphotos as reliable and comprehensive as possible.  The test suite currently has over 400 tests--but there are still some [bugs](https://github.com/RhetTbull/osxphotos/issues?q=is%3Aissue+is%3Aopen+label%3Abug) or incomplete features lurking.  If you find bugs please open an [issue](https://github.com/RhetTbull/osxphotos/issues).  Notable issues include:
+My goal is make osxphotos as reliable and comprehensive as possible.  The test suite currently has over 600 tests--but there are still some [bugs](https://github.com/RhetTbull/osxphotos/issues?q=is%3Aissue+is%3Aopen+label%3Abug) or incomplete features lurking.  If you find bugs please open an [issue](https://github.com/RhetTbull/osxphotos/issues).  Notable issues include:
 
-- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos.  osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`.  If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored.  See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101) Note: Beta version of fix for this bug is implemented in the current version of osxphotos.
-- The `--download-missing` option for `osxphotos export` does not work correctly with burst images.  It will download the primary image but not the other burst images.  See [Issue #75](https://github.com/RhetTbull/osxphotos/issues/75)
+- Face coordinates (mouth, left eye, right eye) may not be correct for images where the head is tilted.  See [Issue #196](https://github.com/RhetTbull/osxphotos/issues/196).
+- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos.  osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`.  If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored.  See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101). Note: Beta version of fix for this bug is implemented in the current version of osxphotos.
+- The `--download-missing` option for `osxphotos export` does not work correctly with burst images.  It will download the primary image but not the other burst images.  See [Issue #75](https://github.com/RhetTbull/osxphotos/issues/75).
 
 ## Implementation Notes
 
@@ -1670,7 +1897,7 @@ This package works by creating a copy of the sqlite3 database that photos uses t
 
 If apple changes the database format this will likely break.
 
-Apple does provide a framework ([PhotoKit](https://developer.apple.com/documentation/photokit?language=objc)) for querying the user's Photos library and I attempted to create the funcationality in this package using this framework but unfortunately PhotoKit does not provide access to much of the needed metadata (such as Faces/Persons) and Apple's System Integrity Protection (SIP) made the interface unreliable.  If you'd like to experiment with the PhotoKit interface, here's some sample [code](https://gist.github.com/RhetTbull/41cc85e5bdeb30f761147ce32fba5c94).  While copying the sqlite file is a bit kludgy, it allows osxphotos to provide access to all available metadata.
+Apple does provide a framework ([PhotoKit](https://developer.apple.com/documentation/photokit?language=objc)) for querying the user's Photos library and I attempted to create the functionality in this package using this framework but unfortunately PhotoKit does not provide access to much of the needed metadata (such as Faces/Persons) and Apple's System Integrity Protection (SIP) made the interface unreliable.  If you'd like to experiment with the PhotoKit interface, here's some sample [code](https://gist.github.com/RhetTbull/41cc85e5bdeb30f761147ce32fba5c94).  While copying the sqlite file is a bit kludgy, it allows osxphotos to provide access to all available metadata.
 
 For additional details about how osxphotos is implemented or if you would like to extend the code, see the [wiki](https://github.com/RhetTbull/osxphotos/wiki).
 

examples/export_by_album.py

@@ -79,7 +79,7 @@ def export(export_path, default_album, library_path, edited):
                 exported = p.export(dest_dir, filename)
                 click.echo(f"Exported {filename} to {exported}")
         else:
-            click.echo(f"Skipping missing photo: {p.original_filename} in album {album}")
+            click.echo(f"Skipping missing photo: {p.original_filename}")
 
 
 if __name__ == "__main__":

examples/export_faces.py

@@ -0,0 +1,83 @@
+""" Export all photos that contain a detected face and draw rectangles around each face
+    photos with no persons/detected faces will not be export 
+
+    This shows how to use the FaceInfo class and is useful for validating that FaceInfo is
+    correctly handling faces.
+
+    To use this, you'll need to install Pillow:
+    python3 -m pip install Pillow
+"""
+
+import os
+
+import click
+from PIL import Image, ImageDraw
+
+import osxphotos
+
+
+@click.command()
+@click.argument("export-path", type=click.Path(exists=True))
+@click.option(
+    "--uuid",
+    metavar="UUID",
+    help="Limit export to optional UUID(s)",
+    required=False,
+    multiple=True,
+)
+@click.option(
+    "--library-path",
+    metavar="PATH",
+    help="Path to Photos library, default to last used library",
+    default=None,
+)
+def export(export_path, library_path, uuid):
+    """ export photos to export_path and draw faces """
+    library_path = os.path.expanduser(library_path) if library_path else None
+    if library_path is not None:
+        photosdb = osxphotos.PhotosDB(library_path)
+    else:
+        photosdb = osxphotos.PhotosDB()
+
+    photos = photosdb.photos(uuid=uuid) if uuid else photosdb.photos(movies=False)
+    for p in photos:
+        if p.person_info and not p.ismissing:
+            # has persons and not missing
+            if "heic" in p.filename.lower():
+                print(f"skipping heic image {p.filename}")
+                continue
+            print(f"exporting photo {p.original_filename}, uuid = {p.uuid}")
+            export = p.export(export_path, p.original_filename, edited=p.hasadjustments)
+            if export:
+                im = Image.open(export[0])
+                draw = ImageDraw.Draw(im)
+                for face in p.face_info:
+                    coords = face.face_rect()
+                    draw.rectangle(coords, width=3)
+                    draw.ellipse(get_circle_points(face.center, 3), width=1)
+                    draw.text(face.mouth, "M", fill=(255, 255, 255, 255))
+                    draw.text(face.left_eye, "L", fill=(255, 255, 255, 255))
+                    draw.text(face.right_eye, "R", fill=(255, 255, 255, 255))
+                im.save(export[0])
+            else:
+                print(f"no photos exported for {p.uuid}")
+
+
+def get_circle_points(xy, radius):
+    """ Returns tuples of (x0, y0), (x1, y1) for a circle centered at x, y with radius
+
+    Arguments:
+        xy: tuple of x, y coordinates
+        radius: radius of circle to draw
+
+    Returns:
+        [(x0, y0), (x1, y1)] for bounding box of circle centered at x, y
+    """
+    x, y = xy
+    x0, y0 = x - radius, y - radius
+    x1, y1 = x + radius, y + radius
+    return [(x0, y0), (x1, y1)]
+
+
+if __name__ == "__main__":
+    export()  # pylint: disable=no-value-for-parameter

examples/force_download.py

@@ -0,0 +1,42 @@
+""" use osxphotos to force the download of photos from iCloud
+    downloads images to a temporary directory then deletes them
+    resulting in the photo being downloaded to Photos library
+"""
+
+import os
+import sys
+import tempfile
+
+import osxphotos
+
+
+def main():
+    photosdb = osxphotos.PhotosDB()
+    tempdir = tempfile.TemporaryDirectory()
+    photos = photosdb.photos()
+    downloaded = 0
+    missing = [photo for photo in photos if photo.ismissing and not photo.shared]
+
+    if not missing:
+        print(f"Did not find any missing photos to download")
+        sys.exit(0)
+
+    print(f"Downloading {len(missing)} photos")
+    for photo in missing:
+        if photo.ismissing:
+            print(f"Downloading photo {photo.original_filename}")
+            downloaded += 1
+            exported = photo.export(tempdir.name, use_photos_export=True, timeout=300)
+            if photo.hasadjustments:
+                exported.extend(
+                    photo.export(tempdir.name, use_photos_export=True, edited=True, timeout=300)
+                )
+            for filename in exported:
+                print(f"Removing temporary file {filename}")
+                os.unlink(filename)
+    print(f"Downloaded {downloaded} photos")
+    tempdir.cleanup()
+
+
+if __name__ == "__main__":
+    main()

examples/photos_repl.py

@@ -58,5 +58,6 @@ if __name__ == "__main__":
     print("getting photos")
     tic = time.perf_counter()
     photos = photosdb.photos(images=True, movies=True)
+    photos.extend(photosdb.photos(images=True, movies=True, intrash=True))
     toc = time.perf_counter()
     print(f"found {len(photos)} photos in {toc-tic} seconds")

osxphotos/__main__.py

@@ -10,6 +10,7 @@ import pathlib
 import pprint
 import sys
 import time
+import unicodedata
 
 import click
 import yaml
@@ -22,7 +23,12 @@ from pathvalidate import (
 
 import osxphotos
 
-from ._constants import _EXIF_TOOL_URL, _PHOTOS_4_VERSION, _UNKNOWN_PLACE
+from ._constants import (
+    _EXIF_TOOL_URL,
+    _PHOTOS_4_VERSION,
+    _UNKNOWN_PLACE,
+    UNICODE_FORMAT,
+)
 from ._export_db import ExportDB, ExportDBInMemory
 from ._version import __version__
 from .datetime_formatter import DateTimeFormatter
@@ -40,10 +46,24 @@ OSXPHOTOS_EXPORT_DB = ".osxphotos_export.db"
 
 
 def verbose(*args, **kwargs):
+    """ print output if verbose flag set """
     if VERBOSE:
         click.echo(*args, **kwargs)
 
 
+def normalize_unicode(value):
+    """ normalize unicode data """
+    if value is not None:
+        if isinstance(value, tuple):
+            return tuple(unicodedata.normalize(UNICODE_FORMAT, v) for v in value)
+        elif isinstance(value, str):
+            return unicodedata.normalize(UNICODE_FORMAT, value)
+        else:
+            return value
+    else:
+        return None
+
+
 def get_photos_db(*db_options):
     """ Return path to photos db, select first non-None db_options
         If no db_options are non-None, try to find library to use in
@@ -77,6 +97,20 @@ def get_photos_db(*db_options):
         return None
 
 
+class DateTimeISO8601(click.ParamType):
+
+    name = "DATETIME"
+
+    def convert(self, value, param, ctx):
+        try:
+            return datetime.datetime.fromisoformat(value)
+        except:
+            self.fail(
+                f"Invalid value for --{param.name}: invalid datetime format {value}. "
+                "Valid format: YYYY-MM-DD[*HH[:MM[:SS[.fff[fff]]]][+HH:MM[:SS[.ffffff]]]]"
+            )
+
+
 # Click CLI object & context settings
 class CLI_Obj:
     def __init__(self, db=None, json=False, debug=False):
@@ -298,6 +332,15 @@ def query_options(f):
             multiple=True,
             help="Search for photos with UUID(s).",
         ),
+        o(
+            "--uuid-from-file",
+            metavar="FILE",
+            default=None,
+            multiple=False,
+            help="Search for photos with UUID(s) loaded from FILE. "
+            "Format is a single UUID per line.  Lines preceeded with # are ignored.",
+            type=click.Path(exists=True),
+        ),
         o(
             "--title",
             metavar="TITLE",
@@ -445,13 +488,13 @@ def query_options(f):
         ),
         o(
             "--from-date",
-            help="Search by start item date, e.g. 2000-01-12T12:00:00 or 2000-12-31 (ISO 8601 w/o TZ).",
-            type=click.DateTime(),
+            help="Search by start item date, e.g. 2000-01-12T12:00:00, 2001-01-12T12:00:00-07:00, or 2000-12-31 (ISO 8601).",
+            type=DateTimeISO8601(),
         ),
         o(
             "--to-date",
-            help="Search by end item date, e.g. 2000-01-12T12:00:00 or 2000-12-31 (ISO 8601 w/o TZ).",
-            type=click.DateTime(),
+            help="Search by end item date, e.g. 2000-01-12T12:00:00, 2001-01-12T12:00:00-07:00, or 2000-12-31 (ISO 8601).",
+            type=DateTimeISO8601(),
         ),
     ]
     for o in options[::-1]:
@@ -520,10 +563,14 @@ def debug_dump(ctx, cli_obj, db, photos_library, dump, uuid):
             print("_dbkeywords_uuid:")
             pprint.pprint(photosdb._dbkeywords_uuid)
         elif attr == "persons":
-            print("_dbfaces_person:")
-            pprint.pprint(photosdb._dbfaces_person)
             print("_dbfaces_uuid:")
             pprint.pprint(photosdb._dbfaces_uuid)
+            print("_dbfaces_pk:")
+            pprint.pprint(photosdb._dbfaces_pk)
+            print("_dbpersons_pk:")
+            pprint.pprint(photosdb._dbpersons_pk)
+            print("_dbpersons_fullname:")
+            pprint.pprint(photosdb._dbpersons_fullname)
         elif attr == "photos":
             if uuid:
                 for uuid_ in uuid:
@@ -565,9 +612,9 @@ def keywords(ctx, cli_obj, db, json_, photos_library):
     photosdb = osxphotos.PhotosDB(dbfile=db)
     keywords = {"keywords": photosdb.keywords_as_dict}
     if json_ or cli_obj.json:
-        click.echo(json.dumps(keywords))
+        click.echo(json.dumps(keywords, ensure_ascii=False))
     else:
-        click.echo(yaml.dump(keywords, sort_keys=False))
+        click.echo(yaml.dump(keywords, sort_keys=False, allow_unicode=True))
 
 
 @cli.command()
@@ -594,9 +641,9 @@ def albums(ctx, cli_obj, db, json_, photos_library):
         albums["shared albums"] = photosdb.albums_shared_as_dict
 
     if json_ or cli_obj.json:
-        click.echo(json.dumps(albums))
+        click.echo(json.dumps(albums, ensure_ascii=False))
     else:
-        click.echo(yaml.dump(albums, sort_keys=False))
+        click.echo(yaml.dump(albums, sort_keys=False, allow_unicode=True))
 
 
 @cli.command()
@@ -620,9 +667,9 @@ def persons(ctx, cli_obj, db, json_, photos_library):
     photosdb = osxphotos.PhotosDB(dbfile=db)
     persons = {"persons": photosdb.persons_as_dict}
     if json_ or cli_obj.json:
-        click.echo(json.dumps(persons))
+        click.echo(json.dumps(persons, ensure_ascii=False))
     else:
-        click.echo(yaml.dump(persons, sort_keys=False))
+        click.echo(yaml.dump(persons, sort_keys=False, allow_unicode=True))
 
 
 @cli.command()
@@ -646,9 +693,9 @@ def labels(ctx, cli_obj, db, json_, photos_library):
     photosdb = osxphotos.PhotosDB(dbfile=db)
     labels = {"labels": photosdb.labels_as_dict}
     if json_ or cli_obj.json:
-        click.echo(json.dumps(labels))
+        click.echo(json.dumps(labels, ensure_ascii=False))
     else:
-        click.echo(yaml.dump(labels, sort_keys=False))
+        click.echo(yaml.dump(labels, sort_keys=False, allow_unicode=True))
 
 
 @cli.command()
@@ -706,9 +753,9 @@ def info(ctx, cli_obj, db, json_, photos_library):
     info["persons"] = persons
 
     if cli_obj.json or json_:
-        click.echo(json.dumps(info))
+        click.echo(json.dumps(info, ensure_ascii=False))
     else:
-        click.echo(yaml.dump(info, sort_keys=False))
+        click.echo(yaml.dump(info, sort_keys=False, allow_unicode=True))
 
 
 @cli.command()
@@ -756,9 +803,9 @@ def places(ctx, cli_obj, db, json_, photos_library):
     # below needed for to make CliRunner work for testing
     cli_json = cli_obj.json if cli_obj is not None else None
     if json_ or cli_json:
-        click.echo(json.dumps(places))
+        click.echo(json.dumps(places, ensure_ascii=False))
     else:
-        click.echo(yaml.dump(places, sort_keys=False))
+        click.echo(yaml.dump(places, sort_keys=False, allow_unicode=True))
 
 
 @cli.command()
@@ -821,7 +868,7 @@ def _list_libraries(json_=False, error=True):
             "system_library": sys_lib,
             "last_library": last_lib,
         }
-        click.echo(json.dumps(libs))
+        click.echo(json.dumps(libs, ensure_ascii=False))
     else:
         last_lib_flag = sys_lib_flag = False
 
@@ -887,6 +934,7 @@ def query(
     album,
     folder,
     uuid,
+    uuid_from_file,
     title,
     no_title,
     description,
@@ -950,6 +998,7 @@ def query(
         album,
         folder,
         uuid,
+        uuid_from_file,
         edited,
         external_edit,
         uti,
@@ -994,6 +1043,12 @@ def query(
     if only_photos:
         ismovie = False
 
+    # load UUIDs if necessary and append to any uuids passed with --uuid
+    if uuid_from_file:
+        uuid_list = list(uuid)  # Click option is a tuple
+        uuid_list.extend(load_uuid_from_file(uuid_from_file))
+        uuid = tuple(uuid_list)
+
     # below needed for to make CliRunner work for testing
     cli_db = cli_obj.db if cli_obj is not None else None
     db = get_photos_db(*photos_library, db, cli_db)
@@ -1086,6 +1141,11 @@ def query(
     help="Hardlink files instead of copying them.  "
     "Cannot be used with --exiftool which creates copies of the files with embedded EXIF data.",
 )
+@click.option(
+    "--touch-file",
+    is_flag=True,
+    help="Sets the file's modification time to match photo date.",
+)
 @click.option(
     "--overwrite",
     is_flag=True,
@@ -1105,6 +1165,11 @@ def query(
     is_flag=True,
     help="Do not export edited version of photo if an edited version exists.",
 )
+@click.option(
+    "--skip-original-if-edited",
+    is_flag=True,
+    help="Do not export original if there is an edited version (exports only the edited version).",
+)
 @click.option(
     "--skip-bursts",
     is_flag=True,
@@ -1146,6 +1211,18 @@ def query(
     '--keyword-template "{created.year}" '
     "See Templating System below.",
 )
+@click.option(
+    "--description-template",
+    metavar="TEMPLATE",
+    multiple=False,
+    default=None,
+    help="For use with --exiftool, --sidecar; specify a template string to use as "
+    "description in the form '{name,DEFAULT}' "
+    "This is the same format as --directory.  For example, if you wanted to append "
+    "'exported with osxphotos on [today's date]' to the description, you could specify "
+    '--description-template "{descr} exported with osxphotos on {today.date}" '
+    "See Templating System below.",
+)
 @click.option(
     "--current-name",
     is_flag=True,
@@ -1217,6 +1294,13 @@ def query(
     "to a filesystem that doesn't support Mac OS extended attributes.  Only use this if you get "
     "an error while exporting.",
 )
+@click.option(
+    "--use-photos-export",
+    is_flag=True,
+    default=False,
+    hidden=True,
+    help="Force the use of AppleScript to export even if not missing (see also --download-missing).",
+)
 @DB_ARGUMENT
 @click.argument("dest", nargs=1, type=click.Path(exists=True))
 @click.pass_obj
@@ -1231,6 +1315,7 @@ def export(
     album,
     folder,
     uuid,
+    uuid_from_file,
     title,
     no_title,
     description,
@@ -1251,15 +1336,18 @@ def export(
     update,
     dry_run,
     export_as_hardlink,
+    touch_file,
     overwrite,
     export_by_date,
     skip_edited,
+    skip_original_if_edited,
     skip_bursts,
     skip_live,
     skip_raw,
     person_keyword,
     album_keyword,
     keyword_template,
+    description_template,
     current_name,
     sidecar,
     only_photos,
@@ -1295,6 +1383,7 @@ def export(
     label,
     deleted,
     deleted_only,
+    use_photos_export,
 ):
     """ Export photos from the Photos database.
         Export path DEST is required.
@@ -1312,7 +1401,7 @@ def export(
     VERBOSE = True if verbose_ else False
 
     if not os.path.isdir(dest):
-        sys.exit("DEST must be valid path")
+        sys.exit(f"DEST {dest} must be valid path")
 
     # sanity check input args
     exclusive = [
@@ -1334,6 +1423,7 @@ def export(
         (export_as_hardlink, exiftool),
         (any(place), no_place),
         (deleted, deleted_only),
+        (skip_edited, skip_original_if_edited),
     ]
     if any(all(bb) for bb in exclusive):
         click.echo("Incompatible export options", err=True)
@@ -1364,6 +1454,12 @@ def export(
     if only_photos:
         ismovie = False
 
+    # load UUIDs if necessary and append to any uuids passed with --uuid
+    if uuid_from_file:
+        uuid_list = list(uuid)  # Click option is a tuple
+        uuid_list.extend(load_uuid_from_file(uuid_from_file))
+        uuid = tuple(uuid_list)
+
     # below needed for to make CliRunner work for testing
     cli_db = cli_obj.db if cli_obj is not None else None
     db = get_photos_db(*photos_library, db, cli_db)
@@ -1459,7 +1555,6 @@ def export(
         deleted_only=deleted_only,
     )
 
-    results_exported = []
     if photos:
         if export_bursts:
             # add the burst_photos to the export set
@@ -1478,10 +1573,12 @@ def export(
         # because the original code used --original-name as an option
         original_name = not current_name
 
+        results_exported = []
         results_new = []
         results_updated = []
         results_skipped = []
         results_exif_updated = []
+        results_touched = []
         if verbose_:
             for p in photos:
                 results = export_photo(
@@ -1494,6 +1591,7 @@ def export(
                     export_as_hardlink=export_as_hardlink,
                     overwrite=overwrite,
                     export_edited=export_edited,
+                    skip_original_if_edited=skip_original_if_edited,
                     original_name=original_name,
                     export_live=export_live,
                     download_missing=download_missing,
@@ -1505,16 +1603,20 @@ def export(
                     album_keyword=album_keyword,
                     person_keyword=person_keyword,
                     keyword_template=keyword_template,
+                    description_template=description_template,
                     export_db=export_db,
                     fileutil=fileutil,
                     dry_run=dry_run,
+                    touch_file=touch_file,
                     edited_suffix=edited_suffix,
+                    use_photos_export=use_photos_export,
                 )
                 results_exported.extend(results.exported)
                 results_new.extend(results.new)
                 results_updated.extend(results.updated)
                 results_skipped.extend(results.skipped)
                 results_exif_updated.extend(results.exif_updated)
+                results_touched.extend(results.touched)
 
         else:
             # show progress bar
@@ -1530,6 +1632,7 @@ def export(
                         export_as_hardlink=export_as_hardlink,
                         overwrite=overwrite,
                         export_edited=export_edited,
+                        skip_original_if_edited=skip_original_if_edited,
                         original_name=original_name,
                         export_live=export_live,
                         download_missing=download_missing,
@@ -1541,34 +1644,42 @@ def export(
                         album_keyword=album_keyword,
                         person_keyword=person_keyword,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                         export_db=export_db,
                         fileutil=fileutil,
                         dry_run=dry_run,
+                        touch_file=touch_file,
                         edited_suffix=edited_suffix,
+                        use_photos_export=use_photos_export,
                     )
                     results_exported.extend(results.exported)
                     results_new.extend(results.new)
                     results_updated.extend(results.updated)
                     results_skipped.extend(results.skipped)
                     results_exif_updated.extend(results.exif_updated)
+                    results_touched.extend(results.touched)
         stop_time = time.perf_counter()
         # print summary results
         if update:
             photo_str_new = "photos" if len(results_new) != 1 else "photo"
-            photo_str_updated = "photos" if len(results_new) != 1 else "photo"
+            photo_str_updated = "photos" if len(results_updated) != 1 else "photo"
             photo_str_skipped = "photos" if len(results_skipped) != 1 else "photo"
             photo_str_exif_updated = (
                 "photos" if len(results_exif_updated) != 1 else "photo"
             )
-            click.echo(
+            summary = (
                 f"Exported: {len(results_new)} {photo_str_new}, "
-                + f"updated: {len(results_updated)} {photo_str_updated}, "
-                + f"skipped: {len(results_skipped)} {photo_str_skipped}, "
-                + f"updated EXIF data: {len(results_exif_updated)} {photo_str_exif_updated}"
+                f"updated: {len(results_updated)} {photo_str_updated}, "
+                f"skipped: {len(results_skipped)} {photo_str_skipped}, "
+                f"updated EXIF data: {len(results_exif_updated)} {photo_str_exif_updated}"
             )
         else:
             photo_str = "photos" if len(results_exported) != 1 else "photo"
-            click.echo(f"Exported: {len(results_exported)} {photo_str}")
+            summary = f"Exported: {len(results_exported)} {photo_str}"
+        photo_str_touched = "photos" if len(results_touched) != 1 else "photo"
+        if touch_file:
+            summary += f", touched date: {len(results_touched)} {photo_str_touched}"
+        click.echo(summary)
         click.echo(f"Elapsed time: {(stop_time-start_time):.3f} seconds")
     else:
         click.echo("Did not find any photos to export")
@@ -1777,6 +1888,15 @@ def _query(
             to_date=to_date,
         )
 
+    person = normalize_unicode(person)
+    keyword = normalize_unicode(keyword)
+    album = normalize_unicode(album)
+    folder = normalize_unicode(folder)
+    title = normalize_unicode(title)
+    description = normalize_unicode(description)
+    place = normalize_unicode(place)
+    label = normalize_unicode(label)
+
     if album:
         photos = get_photos_by_attribute(photos, "albums", album, ignore_case)
 
@@ -2001,6 +2121,7 @@ def export_photo(
     export_as_hardlink=None,
     overwrite=None,
     export_edited=None,
+    skip_original_if_edited=None,
     original_name=None,
     export_live=None,
     download_missing=None,
@@ -2012,10 +2133,13 @@ def export_photo(
     album_keyword=None,
     person_keyword=None,
     keyword_template=None,
+    description_template=None,
     export_db=None,
     fileutil=FileUtil,
     dry_run=None,
+    touch_file=None,
     edited_suffix="_edited",
+    use_photos_export=False,
 ):
     """ Helper function for export that does the actual export
 
@@ -2036,12 +2160,17 @@ def export_photo(
         filename_template: template use to determine output file
         no_extended_attributes: boolean; if True, exports photo without preserving extended attributes
         export_raw: boolean; if True exports RAW image associate with the photo
+        export_edited: boolean; if True exports edited version of photo if there is one
+        skip_original_if_edited: boolean; if True does not export original if photo has been edited
         album_keyword: boolean; if True, exports album names as keywords in metadata
         person_keyword: boolean; if True, exports person names as keywords in metadata
         keyword_template: list of strings; if provided use rendered template strings as keywords
+        description_template: string; optional template string that will be rendered for use as photo description
         export_db: export database instance compatible with ExportDB_ABC
         fileutil: file util class compatible with FileUtilABC
         dry_run: boolean; if True, doesn't actually export or update any files
+        touch_file: boolean; sets file's modification time to match photo date
+        use_photos_export: boolean; if True forces the use of AppleScript to export even if photo not missing
 
     Returns:
         list of path(s) of exported photo or None if photo was missing
@@ -2055,24 +2184,33 @@ def export_photo(
     if not download_missing:
         if photo.ismissing:
             space = " " if not verbose_ else ""
-            verbose(f"{space}Skipping missing photo {photo.filename}")
-            return ExportResults([], [], [], [], [])
+            verbose(f"{space}Skipping missing photo {photo.original_filename}")
+            return ExportResults([], [], [], [], [], [])
         elif not os.path.exists(photo.path):
             space = " " if not verbose_ else ""
             verbose(
                 f"{space}WARNING: file {photo.path} is missing but ismissing=False, "
-                f"skipping {photo.filename}"
+                f"skipping {photo.original_filename}"
             )
-            return ExportResults([], [], [], [], [])
+            return ExportResults([], [], [], [], [], [])
     elif photo.ismissing and not photo.iscloudasset or not photo.incloud:
         verbose(
-            f"Skipping missing {photo.filename}: not iCloud asset or missing from cloud"
+            f"Skipping missing {photo.original_filename}: not iCloud asset or missing from cloud"
         )
-        return ExportResults([], [], [], [], [])
+        return ExportResults([], [], [], [], [], [])
+
+    results_exported = []
+    results_new = []
+    results_updated = []
+    results_skipped = []
+    results_exif_updated = []
+    results_touched = []
+
+    export_original = not (skip_original_if_edited and photo.hasadjustments)
 
     filenames = get_filenames_from_template(photo, filename_template, original_name)
     for filename in filenames:
-        verbose(f"Exporting {photo.filename} as {filename}")
+        verbose(f"Exporting {photo.original_filename} ({photo.filename}) as {filename}")
 
         dest_paths = get_dirnames_from_template(
             photo, directory, export_by_date, dest, dry_run
@@ -2087,60 +2225,64 @@ def export_photo(
 
         # if download_missing and the photo is missing or path doesn't exist,
         # try to download with Photos
-        use_photos_export = download_missing and (
-            photo.ismissing or not os.path.exists(photo.path)
+        use_photos_export = (
+            download_missing and (photo.ismissing or not os.path.exists(photo.path))
+            if not use_photos_export
+            else True
         )
 
         # export the photo to each path in dest_paths
-        results_exported = []
-        results_new = []
-        results_updated = []
-        results_skipped = []
-        results_exif_updated = []
         for dest_path in dest_paths:
-            export_results = photo.export2(
-                dest_path,
-                filename,
-                sidecar_json=sidecar_json,
-                sidecar_xmp=sidecar_xmp,
-                live_photo=export_live,
-                raw_photo=export_raw,
-                export_as_hardlink=export_as_hardlink,
-                overwrite=overwrite,
-                use_photos_export=use_photos_export,
-                exiftool=exiftool,
-                no_xattr=no_extended_attributes,
-                use_albums_as_keywords=album_keyword,
-                use_persons_as_keywords=person_keyword,
-                keyword_template=keyword_template,
-                update=update,
-                export_db=export_db,
-                fileutil=fileutil,
-                dry_run=dry_run,
-            )
+            if not export_original:
+                verbose(f"Skipping original version of {photo.original_filename}")
+            else:
+                export_results = photo.export2(
+                    dest_path,
+                    filename,
+                    sidecar_json=sidecar_json,
+                    sidecar_xmp=sidecar_xmp,
+                    live_photo=export_live,
+                    raw_photo=export_raw,
+                    export_as_hardlink=export_as_hardlink,
+                    overwrite=overwrite,
+                    use_photos_export=use_photos_export,
+                    exiftool=exiftool,
+                    no_xattr=no_extended_attributes,
+                    use_albums_as_keywords=album_keyword,
+                    use_persons_as_keywords=person_keyword,
+                    keyword_template=keyword_template,
+                    description_template=description_template,
+                    update=update,
+                    export_db=export_db,
+                    fileutil=fileutil,
+                    dry_run=dry_run,
+                    touch_file=touch_file,
+                )
 
-            results_exported.extend(export_results.exported)
-            results_new.extend(export_results.new)
-            results_updated.extend(export_results.updated)
-            results_skipped.extend(export_results.skipped)
-            results_exif_updated.extend(export_results.exif_updated)
+                results_exported.extend(export_results.exported)
+                results_new.extend(export_results.new)
+                results_updated.extend(export_results.updated)
+                results_skipped.extend(export_results.skipped)
+                results_exif_updated.extend(export_results.exif_updated)
+                results_touched.extend(export_results.touched)
 
-            if verbose_:
-                for exported in export_results.exported:
-                    verbose(f"Exported {exported}")
-                for new in export_results.new:
-                    verbose(f"Exported new file {new}")
-                for updated in export_results.updated:
-                    verbose(f"Exported updated file {updated}")
-                for skipped in export_results.skipped:
-                    verbose(f"Skipped up to date file {skipped}")
+                if verbose_:
+                    for exported in export_results.exported:
+                        verbose(f"Exported {exported}")
+                    for new in export_results.new:
+                        verbose(f"Exported new file {new}")
+                    for updated in export_results.updated:
+                        verbose(f"Exported updated file {updated}")
+                    for skipped in export_results.skipped:
+                        verbose(f"Skipped up to date file {skipped}")
+                    for touched in export_results.touched:
+                        verbose(f"Touched date on file {touched}")
 
             # if export-edited, also export the edited version
             # verify the photo has adjustments and valid path to avoid raising an exception
             if export_edited and photo.hasadjustments:
                 # if download_missing and the photo is missing or path doesn't exist,
                 # try to download with Photos
-                use_photos_export = download_missing and photo.path_edited is None
                 if not download_missing and photo.path_edited is None:
                     verbose(f"Skipping missing edited photo for {filename}")
                 else:
@@ -2168,10 +2310,12 @@ def export_photo(
                         use_albums_as_keywords=album_keyword,
                         use_persons_as_keywords=person_keyword,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                         update=update,
                         export_db=export_db,
                         fileutil=fileutil,
                         dry_run=dry_run,
+                        touch_file=touch_file,
                     )
 
                     results_exported.extend(export_results_edited.exported)
@@ -2179,6 +2323,7 @@ def export_photo(
                     results_updated.extend(export_results_edited.updated)
                     results_skipped.extend(export_results_edited.skipped)
                     results_exif_updated.extend(export_results_edited.exif_updated)
+                    results_touched.extend(export_results_edited.touched)
 
                     if verbose_:
                         for exported in export_results_edited.exported:
@@ -2189,6 +2334,8 @@ def export_photo(
                             verbose(f"Exported updated file {updated}")
                         for skipped in export_results_edited.skipped:
                             verbose(f"Skipped up to date file {skipped}")
+                        for touched in export_results_edited.touched:
+                            verbose(f"Touched date on file {touched}")
 
     return ExportResults(
         results_exported,
@@ -2196,6 +2343,7 @@ def export_photo(
         results_updated,
         results_skipped,
         results_exif_updated,
+        results_touched,
     )
 
 
@@ -2316,5 +2464,32 @@ def find_files_in_branch(pathname, filename):
     return files
 
 
+def load_uuid_from_file(filename):
+    """ Load UUIDs from file.  Does not validate UUIDs.
+        Format is 1 UUID per line, any line beginning with # is ignored.
+        Whitespace is stripped.
+
+    Arguments:
+        filename: file name of the file containing UUIDs
+    
+    Returns:
+        list of UUIDs or empty list of no UUIDs in file
+    
+    Raises:
+        FileNotFoundError if file does not exist
+    """
+
+    if not pathlib.Path(filename).is_file():
+        raise FileNotFoundError(f"Could not find file {filename}")
+
+    uuid = []
+    with open(filename, "r") as uuid_file:
+        for line in uuid_file:
+            line = line.strip()
+            if len(line) and line[0] != "#":
+                uuid.append(line)
+    return uuid
+
+
 if __name__ == "__main__":
     cli()  # pylint: disable=no-value-for-parameter

osxphotos/_constants.py

@@ -3,6 +3,14 @@ Constants used by osxphotos
 """
 
 import os.path
+from datetime import datetime
+
+# Time delta: add this to Photos times to get unix time
+# Apple Epoch is Jan 1, 2001
+TIME_DELTA = (datetime(2001, 1, 1, 0, 0) - datetime(1970, 1, 1, 0, 0)).total_seconds()
+
+# Unicode format to use for comparing strings
+UNICODE_FORMAT = "NFC"
 
 # which Photos library database versions have been tested
 # Photos 2.0 (10.12.6) == 2622
@@ -10,18 +18,48 @@ import os.path
 # Photos 4.0 (10.14.5) == 4016
 # Photos 4.0 (10.14.6) == 4025
 # Photos 5.0 (10.15.0) == 6000
-# TODO: Should this also use compatibleBackToVersion from LiGlobals?
 _TESTED_DB_VERSIONS = ["6000", "4025", "4016", "3301", "2622"]
 
+# database model versions (applies to Photos 5, Photos 6)
+# these come from PLModelVersion key in binary plist in Z_METADATA.Z_PLIST
+# Photos 5 (10.15.1) == 13537
+# Photos 5 (10.15.4, 10.15.5, 10.15.6) == 13703
+# Photos 6 (10.16.0 Beta) == 14104
+_TEST_MODEL_VERSIONS = ["13537", "13703", "14104"]
+
 # only version 3 - 4 have RKVersion.selfPortrait
 _PHOTOS_3_VERSION = "3301"
 
 # versions 5.0 and later have a different database structure
 _PHOTOS_4_VERSION = "4025"  # latest Mojove version on 10.14.6
-_PHOTOS_5_VERSION = "6000"  # seems to be current on 10.15.1 through 10.15.5
+_PHOTOS_5_VERSION = "6000"  # seems to be current on 10.15.1 through 10.15.6
+
+# Ranges for model version by Photos version
+_PHOTOS_5_MODEL_VERSION = [13000, 13999]
+_PHOTOS_6_MODEL_VERSION = [14000, 14999]
+
+# some table names differ between Photos 5 and Photos 6
+_DB_TABLE_NAMES = {
+    5: {
+        "ASSET": "ZGENERICASSET",
+        "KEYWORD_JOIN": "Z_1KEYWORDS.Z_37KEYWORDS",
+        "ALBUM_JOIN": "Z_26ASSETS.Z_34ASSETS",
+        "ALBUM_SORT_ORDER": "Z_26ASSETS.Z_FOK_34ASSETS",
+        "IMPORT_FOK": "ZGENERICASSET.Z_FOK_IMPORTSESSION",
+        "DEPTH_STATE": "ZGENERICASSET.ZDEPTHSTATES",
+    },
+    6: {
+        "ASSET": "ZASSET",
+        "KEYWORD_JOIN": "Z_1KEYWORDS.Z_36KEYWORDS",
+        "ALBUM_JOIN": "Z_26ASSETS.Z_3ASSETS",
+        "ALBUM_SORT_ORDER": "Z_26ASSETS.Z_FOK_3ASSETS",
+        "IMPORT_FOK": "null",
+        "DEPTH_STATE": "ZASSET.ZDEPTHTYPE",
+    },
+}
 
 # which major version operating systems have been tested
-_TESTED_OS_VERSIONS = ["12", "13", "14", "15"]
+_TESTED_OS_VERSIONS = ["12", "13", "14", "15", "16"]
 
 # Photos 5 has persons who are empty string if unidentified face
 _UNKNOWN_PERSON = "_UNKNOWN_"
@@ -47,6 +85,7 @@ _PHOTOS_5_ALBUM_KIND = 2  # normal user album
 _PHOTOS_5_SHARED_ALBUM_KIND = 1505  # shared album
 _PHOTOS_5_FOLDER_KIND = 4000  # user folder
 _PHOTOS_5_ROOT_FOLDER_KIND = 3999  # root folder
+_PHOTOS_5_IMPORT_SESSION_ALBUM_KIND = 1506  # import session
 
 _PHOTOS_4_ALBUM_KIND = 3  # RKAlbum.albumSubclass
 _PHOTOS_4_TOP_LEVEL_ALBUM = "TopLevelAlbums"

osxphotos/_export_db.py

@@ -189,7 +189,12 @@ class ExportDB(ExportDB_ABC):
                 (filename,),
             )
             results = c.fetchone()
-            stats = results[0:3] if results else None
+            if results:
+                stats = results[0:3]
+                mtime = int(stats[2]) if stats[2] is not None else None
+                stats = (stats[0], stats[1], mtime)
+            else:
+                stats = (None, None, None)
         except Error as e:
             logging.warning(e)
             stats = (None, None, None)
@@ -232,7 +237,12 @@ class ExportDB(ExportDB_ABC):
                 (filename,),
             )
             results = c.fetchone()
-            stats = results[0:3] if results else None
+            if results:
+                stats = results[0:3]
+                mtime = int(stats[2]) if stats[2] is not None else None
+                stats = (stats[0], stats[1], mtime)
+            else:
+                stats = (None, None, None)
         except Error as e:
             logging.warning(e)
             stats = (None, None, None)

osxphotos/_version.py

@@ -1,3 +1,3 @@
 """ version info """
 
-__version__ = "0.30.2"
+__version__ = "0.34.3"

osxphotos/albuminfo.py

@@ -10,7 +10,7 @@ Represents a single Folder in the Photos library and provides access to the fold
 PhotosDB.folders() returns a list of FolderInfo objects
 """
 
-import logging
+from datetime import datetime, timedelta, timezone
 
 from ._constants import (
     _PHOTOS_4_ALBUM_KIND,
@@ -18,11 +18,34 @@ from ._constants import (
     _PHOTOS_4_VERSION,
     _PHOTOS_5_ALBUM_KIND,
     _PHOTOS_5_FOLDER_KIND,
+    TIME_DELTA,
 )
+from .datetime_utils import get_local_tz
 
 
-class AlbumInfo:
+def sort_list_by_keys(values, sort_keys):
+    """ Sorts list values by a second list sort_keys
+        e.g. given ["a","c","b"], [1, 3, 2], returns ["a", "b", "c"]
+
+    Args:
+        values: a list of values to be sorted
+        sort_keys: a list of keys to sort values by
+    
+    Returns:
+        list of values, sorted by sort_keys
+    
+    Raises:
+        ValueError: raised if len(values) != len(sort_keys)
     """
+    if len(values) != len(sort_keys):
+        return ValueError("values and sort_keys must have same length")
+
+    return list(zip(*sorted(zip(sort_keys, values))))[1]
+
+
+class AlbumInfoBaseClass:
+    """
+    Base class for AlbumInfo, ImportInfo
     Info about a specific Album, contains all the details about the album
     including folders, photos, etc.
     """
@@ -31,25 +54,111 @@ class AlbumInfo:
         self._uuid = uuid
         self._db = db
         self._title = self._db._dbalbum_details[uuid]["title"]
-
-    @property
-    def title(self):
-        """ return title / name of album """
-        return self._title
+        self._creation_date_timestamp = self._db._dbalbum_details[uuid]["creation_date"]
+        self._start_date_timestamp = self._db._dbalbum_details[uuid]["start_date"]
+        self._end_date_timestamp = self._db._dbalbum_details[uuid]["end_date"]
+        self._local_tz = get_local_tz(
+            datetime.fromtimestamp(self._creation_date_timestamp + TIME_DELTA)
+        )
 
     @property
     def uuid(self):
         """ return uuid of album """
         return self._uuid
 
+    @property
+    def creation_date(self):
+        """ return creation date of album """
+        try:
+            return self._creation_date
+        except AttributeError:
+            try:
+                self._creation_date = (
+                    datetime.fromtimestamp(
+                        self._creation_date_timestamp + TIME_DELTA
+                    ).astimezone(tz=self._local_tz)
+                    if self._creation_date_timestamp
+                    else datetime(1970, 1, 1, 0, 0, 0).astimezone(
+                        tz=timezone(timedelta(0))
+                    )
+                )
+            except ValueError:
+                self._creation_date = datetime(1970, 1, 1, 0, 0, 0).astimezone(
+                    tz=timezone(timedelta(0))
+                )
+            return self._creation_date
+
+    @property
+    def start_date(self):
+        """ For Albums, return start date (earliest image) of album or None for albums with no images
+            For Import Sessions, return start date of import session (when import began) """
+        try:
+            return self._start_date
+        except AttributeError:
+            try:
+                self._start_date = (
+                    datetime.fromtimestamp(
+                        self._start_date_timestamp + TIME_DELTA
+                    ).astimezone(tz=self._local_tz)
+                    if self._start_date_timestamp
+                    else None
+                )
+            except ValueError:
+                self._start_date = None
+            return self._start_date
+
+    @property
+    def end_date(self):
+        """ For Albums, return end date (most recent image) of album or None for albums with no images
+            For Import Sessions, return end date of import sessions (when import was completed) """
+        try:
+            return self._end_date
+        except AttributeError:
+            try:
+                self._end_date = (
+                    datetime.fromtimestamp(
+                        self._end_date_timestamp + TIME_DELTA
+                    ).astimezone(tz=self._local_tz)
+                    if self._end_date_timestamp
+                    else None
+                )
+            except ValueError:
+                self._end_date = None
+            return self._end_date
+
     @property
     def photos(self):
-        """ return list of photos contained in album """
+        return []
+
+    def __len__(self):
+        """ return number of photos contained in album """
+        return len(self.photos)
+
+
+class AlbumInfo(AlbumInfoBaseClass):
+    """
+    Base class for AlbumInfo, ImportInfo
+    Info about a specific Album, contains all the details about the album
+    including folders, photos, etc.
+    """
+
+    @property
+    def title(self):
+        """ return title / name of album """
+        return self._title
+
+    @property
+    def photos(self):
+        """ return list of photos contained in album sorted in same sort order as Photos """
         try:
             return self._photos
         except AttributeError:
-            uuid = self._db._dbalbums_album[self._uuid]
-            self._photos = self._db.photos(uuid=uuid)
+            if self.uuid in self._db._dbalbums_album:
+                uuid, sort_order = zip(*self._db._dbalbums_album[self.uuid])
+                sorted_uuid = sort_list_by_keys(uuid, sort_order)
+                self._photos = self._db.photos_by_uuid(sorted_uuid)
+            else:
+                self._photos = []
             return self._photos
 
     @property
@@ -100,9 +209,24 @@ class AlbumInfo:
                 )
             return self._parent
 
-    def __len__(self):
-        """ return number of photos contained in album """
-        return len(self.photos)
+
+class ImportInfo(AlbumInfoBaseClass):
+    @property
+    def photos(self):
+        """ return list of photos contained in import session """
+        try:
+            return self._photos
+        except AttributeError:
+            uuid_list, sort_order = zip(
+                *[
+                    (uuid, self._db._dbphotos[uuid]["fok_import_session"])
+                    for uuid in self._db._dbphotos
+                    if self._db._dbphotos[uuid]["import_uuid"] == self.uuid
+                ]
+            )
+            sorted_uuid = sort_list_by_keys(uuid_list, sort_order)
+            self._photos = self._db.photos_by_uuid(sorted_uuid)
+            return self._photos
 
 
 class FolderInfo:

osxphotos/datetime_utils.py

@@ -0,0 +1,62 @@
+""" datetime utilities """
+
+import datetime
+
+
+def get_local_tz(dt):
+    """ return local timezone as datetime.timezone tzinfo for dt
+    
+    Args:
+        dt: datetime.datetime
+    
+    Returns:
+        local timezone for dt as datetime.timezone
+
+    Raises:
+        ValueError if dt is not timezone naive
+    """
+    if not datetime_has_tz(dt):
+        return dt.astimezone().tzinfo
+    else:
+        raise ValueError("dt must be naive datetime.datetime object")
+
+
+def datetime_remove_tz(dt):
+    """ remove timezone from a datetime.datetime object
+        dt: datetime.datetime object with tzinfo
+        returns: dt without any timezone info (naive datetime object) """
+
+    if type(dt) != datetime.datetime:
+        raise TypeError(f"dt must be type datetime.datetime, not {type(dt)}")
+
+    return dt.replace(tzinfo=None)
+
+
+def datetime_has_tz(dt):
+    """ return True if datetime dt has tzinfo else False
+        dt: datetime.datetime
+        returns True if dt is timezone aware, else False """
+
+    if type(dt) != datetime.datetime:
+        raise TypeError(f"dt must be type datetime.datetime, not {type(dt)}")
+
+    return dt.tzinfo is not None and dt.tzinfo.utcoffset(dt) is not None
+
+
+def datetime_naive_to_local(dt):
+    """ convert naive (timezone unaware) datetime.datetime
+        to aware timezone in local timezone
+        dt: datetime.datetime without timezone
+        returns: datetime.datetime with local timezone """
+
+    if type(dt) != datetime.datetime:
+        raise TypeError(f"dt must be type datetime.datetime, not {type(dt)}")
+
+    if dt.tzinfo is not None and dt.tzinfo.utcoffset(dt) is not None:
+        # has timezone info
+        raise ValueError(
+            "dt must be naive/timezone unaware: "
+            f"{dt} has tzinfo {dt.tzinfo} and offset {dt.tizinfo.utcoffset(dt)}"
+        )
+
+    return dt.replace(tzinfo=get_local_tz(dt))

osxphotos/exiftool.py

@@ -8,6 +8,7 @@
 import json
 import logging
 import os
+import shutil
 import subprocess
 import sys
 from functools import lru_cache  # pylint: disable=syntax-error
@@ -22,8 +23,7 @@ EXIFTOOL_STAYOPEN_EOF_LEN = len(EXIFTOOL_STAYOPEN_EOF)
 @lru_cache(maxsize=1)
 def get_exiftool_path():
     """ return path of exiftool, cache result """
-    result = subprocess.run(["which", "exiftool"], stdout=subprocess.PIPE)
-    exiftool_path = result.stdout.decode("utf-8")
+    exiftool_path = shutil.which('exiftool')
     if _debug():
         logging.debug("exiftool path = %s" % (exiftool_path))
     if exiftool_path:
@@ -98,6 +98,7 @@ class _ExifToolProc:
                 "-",  # read from stdin
                 "-common_args",  # specifies args common to all commands subsequently run
                 "-n",  # no print conversion (e.g. print tag values in machine readable format)
+                "-P",  # Preserve file modification date/time (possible interfere w/ --touch-file)
                 "-G",  # print group name for each tag
             ],
             stdin=subprocess.PIPE,

osxphotos/fileutil.py

@@ -29,7 +29,17 @@ class FileUtilABC(ABC):
 
     @classmethod
     @abstractmethod
-    def cmp_sig(cls, file1, file2):
+    def utime(cls, path, times):
+        pass
+
+    @classmethod
+    @abstractmethod
+    def cmp(cls, file1, file2, mtime1=None):
+        pass
+
+    @classmethod
+    @abstractmethod
+    def cmp_file_sig(cls, file1, file2):
         pass
 
     @classmethod
@@ -104,11 +114,37 @@ class FileUtilMacOS(FileUtilABC):
             os.unlink(filepath)
 
     @classmethod
-    def cmp_sig(cls, f1, s2):
+    def utime(cls, path, times):
+        """ Set the access and modified time of path. """
+        os.utime(path, times)
+
+    @classmethod
+    def cmp(cls, f1, f2, mtime1=None):
+        """Does shallow compare (file signatures) of f1 to file f2.
+        Arguments:
+        f1 --  File name
+        f2 -- File name
+        mtime1 -- optional, pass alternate file modification timestamp for f1; will be converted to int
+
+        Return value:
+        True if the file signatures as returned by stat are the same, False otherwise. 
+        Does not do a byte-by-byte comparison.
+        """
+
+        s1 = cls._sig(os.stat(f1))
+        if mtime1 is not None:
+            s1 = (s1[0], s1[1], int(mtime1))
+        s2 = cls._sig(os.stat(f2))
+        if s1[0] != stat.S_IFREG or s2[0] != stat.S_IFREG:
+            return False
+        return s1 == s2
+
+    @classmethod
+    def cmp_file_sig(cls, f1, s2):
         """Compare file f1 to signature s2.
         Arguments:
         f1 --  File name
-        s2 -- stats as returned by sig
+        s2 -- stats as returned by _sig
 
         Return value:
         True if the files are the same, False otherwise.
@@ -130,7 +166,12 @@ class FileUtilMacOS(FileUtilABC):
 
     @staticmethod
     def _sig(st):
-        return (stat.S_IFMT(st.st_mode), st.st_size, st.st_mtime)
+        """ return tuple of (mode, size, mtime) of file based on os.stat
+            Args:
+                st: os.stat signature
+        """
+        # use int(st.st_mtime) because ditto does not copy fractional portion of mtime
+        return (stat.S_IFMT(st.st_mode), st.st_size, int(st.st_mtime))
 
 
 class FileUtil(FileUtilMacOS):
@@ -141,8 +182,8 @@ class FileUtil(FileUtilMacOS):
 
 class FileUtilNoOp(FileUtil):
     """ No-Op implementation of FileUtil for testing / dry-run mode
-        all methods with exception of cmp_sig and file_cmp are no-op
-        cmp_sig functions as FileUtil.cmp_sig does
+        all methods with exception of cmp, cmp_file_sig and file_cmp are no-op
+        cmp and cmp_file_sig functions as FileUtil methods do
         file_cmp returns mock data
     """
 
@@ -172,6 +213,10 @@ class FileUtilNoOp(FileUtil):
     def unlink(cls, dest):
         cls.verbose(f"unlink: {dest}")
 
+    @classmethod
+    def utime(cls, path, times):
+        cls.verbose(f"utime: {path}, {times}")
+
     @classmethod
     def file_sig(cls, file1):
         cls.verbose(f"file_sig: {file1}")

osxphotos/personinfo.py

@@ -0,0 +1,408 @@
+""" PhotoInfo and FaceInfo classes to expose info about persons and faces in the Photos library """
+
+import json
+import logging
+import math
+
+
+class PersonInfo:
+    """ Info about a person in the Photos library
+    """
+
+    def __init__(self, db=None, pk=None):
+        """ Creates a new PersonInfo instance
+
+        Arguments:
+            db: instance of PhotosDB object
+            pk: primary key value of person to initialize PersonInfo with    
+        
+        Returns:
+            PersonInfo instance
+        """
+        self._db = db
+        self._pk = pk
+
+        person = self._db._dbpersons_pk[pk]
+        self.uuid = person["uuid"]
+        self.name = person["fullname"]
+        self.display_name = person["displayname"]
+        self.keyface = person["keyface"]
+        self.facecount = person["facecount"]
+
+    @property
+    def keyphoto(self):
+        try:
+            return self._keyphoto
+        except AttributeError:
+            person = self._db._dbpersons_pk[self._pk]
+            if person["photo_uuid"]:
+                try:
+                    key_photo = self._db.get_photo(person["photo_uuid"])
+                except IndexError:
+                    key_photo = None
+            else:
+                key_photo = None
+            self._keyphoto = key_photo
+            return self._keyphoto
+
+    @property
+    def photos(self):
+        """ Returns list of PhotoInfo objects associated with this person """
+        return self._db.photos_by_uuid(self._db._dbfaces_pk[self._pk])
+
+    @property
+    def face_info(self):
+        """ Returns a list of FaceInfo objects associated with this person sorted by quality score
+            Highest quality face is result[0] and lowest quality face is result[n] 
+        """
+        try:
+            faces = self._db._db_faceinfo_person[self._pk]
+            return sorted(
+                [FaceInfo(db=self._db, pk=face) for face in faces],
+                key=lambda face: face.quality,
+                reverse=True,
+            )
+        except KeyError:
+            # no faces
+            return []
+
+    def json(self):
+        """ Returns JSON representation of class instance """
+        keyphoto = self.keyphoto.uuid if self.keyphoto is not None else None
+        person = {
+            "uuid": self.uuid,
+            "name": self.name,
+            "displayname": self.display_name,
+            "keyface": self.keyface,
+            "facecount": self.facecount,
+            "keyphoto": keyphoto,
+        }
+        return json.dumps(person)
+
+    def __str__(self):
+        return f"PersonInfo(name={self.name}, display_name={self.display_name}, uuid={self.uuid}, facecount={self.facecount})"
+
+    def __eq__(self, other):
+        if not isinstance(other, type(self)):
+            return False
+
+        return all(
+            getattr(self, field) == getattr(other, field) for field in ["_db", "_pk"]
+        )
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+
+class FaceInfo:
+    """ Info about a face in the Photos library
+    """
+
+    def __init__(self, db=None, pk=None):
+        """ Creates a new FaceInfo instance
+
+        Arguments:
+            db: instance of PhotosDB object
+            pk: primary key value of face to init the object with    
+        
+        Returns:
+            FaceInfo instance
+        """
+        self._db = db
+        self._pk = pk
+
+        face = self._db._db_faceinfo_pk[pk]
+        self._info = face
+        self.uuid = face["uuid"]
+        self.name = face["fullname"]
+        self.asset_uuid = face["asset_uuid"]
+        self._person_pk = face["person"]
+        self.center_x = face["centerx"]
+        self.center_y = face["centery"]
+        self.mouth_x = face["mouthx"]
+        self.mouth_y = face["mouthy"]
+        self.left_eye_x = face["lefteyex"]
+        self.left_eye_y = face["lefteyey"]
+        self.right_eye_x = face["righteyex"]
+        self.right_eye_y = face["righteyey"]
+        self.size = face["size"]
+        self.quality = face["quality"]
+        self.source_width = face["sourcewidth"]
+        self.source_height = face["sourceheight"]
+        self.has_smile = face["has_smile"]
+        self.left_eye_closed = face["left_eye_closed"]
+        self.right_eye_closed = face["right_eye_closed"]
+        self.manual = face["manual"]
+        self.face_type = face["facetype"]
+        self.age_type = face["agetype"]
+        self.bald_type = face["baldtype"]
+        self.eye_makeup_type = face["eyemakeuptype"]
+        self.eye_state = face["eyestate"]
+        self.facial_hair_type = face["facialhairtype"]
+        self.gender_type = face["gendertype"]
+        self.glasses_type = face["glassestype"]
+        self.hair_color_type = face["haircolortype"]
+        self.intrash = face["intrash"]
+        self.lip_makeup_type = face["lipmakeuptype"]
+        self.smile_type = face["smiletype"]
+
+    @property
+    def center(self):
+        """ Coordinates, in PIL format, for center of face 
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point((self.center_x, self.center_y))
+
+    @property
+    def size_pixels(self):
+        """ Size of face in pixels (centered around center_x, center_y)
+
+        Returns:
+            size, in int pixels, of a circle drawn around the center of the face
+        """
+        photo = self.photo
+        size_reference = photo.width if photo.width > photo.height else photo.height
+        return self.size * size_reference
+
+    @property
+    def mouth(self):
+        """ Coordinates, in PIL format, for mouth position
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point_with_rotation((self.mouth_x, self.mouth_y))
+
+    @property
+    def left_eye(self):
+        """ Coordinates, in PIL format, for left eye position
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point_with_rotation((self.left_eye_x, self.left_eye_y))
+
+    @property
+    def right_eye(self):
+        """ Coordinates, in PIL format, for right eye position
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point_with_rotation((self.right_eye_x, self.right_eye_y))
+
+    @property
+    def person_info(self):
+        """ PersonInfo instance for person associated with this face """
+        try:
+            return self._person
+        except AttributeError:
+            self._person = PersonInfo(db=self._db, pk=self._person_pk)
+            return self._person
+
+    @property
+    def photo(self):
+        """ PhotoInfo instance associated with this face """
+        try:
+            return self._photo
+        except AttributeError:
+            self._photo = self._db.get_photo(self.asset_uuid)
+            if self._photo is None:
+                logging.warning(f"Could not get photo for uuid: {self.asset_uuid}")
+            return self._photo
+
+    def face_rect(self):
+        """ Get face rectangle coordinates for current version of the associated image
+            If image has been edited, rectangle applies to edited version, otherwise original version
+            Coordinates in format and reference frame used by PIL
+
+        Returns:
+            list [(x0, x1), (y0, y1)] of coordinates in reference frame used by PIL
+        """
+        photo = self.photo
+        size_reference = photo.width if photo.width > photo.height else photo.height
+        radius = (self.size / 2) * size_reference
+        x, y = self._make_point((self.center_x, self.center_y))
+        x0, y0 = x - radius, y - radius
+        x1, y1 = x + radius, y + radius
+        return [(x0, y0), (x1, y1)]
+
+    def roll_pitch_yaw(self):
+        """ Roll, pitch, yaw of face in radians as tuple """
+        info = self._info
+        roll = 0 if info["roll"] is None else info["roll"]
+        pitch = 0 if info["pitch"] is None else info["pitch"]
+        yaw = 0 if info["yaw"] is None else info["yaw"]
+
+        return (roll, pitch, yaw)
+
+    @property
+    def roll(self):
+        """ Return roll angle in radians of the face region """
+        roll, _, _ = self.roll_pitch_yaw()
+        return roll
+
+    @property
+    def pitch(self):
+        """ Return pitch angle in radians of the face region """
+        _, pitch, _ = self.roll_pitch_yaw()
+        return pitch
+
+    @property
+    def yaw(self):
+        """ Return yaw angle in radians of the face region """
+        _, _, yaw = self.roll_pitch_yaw()
+        return yaw
+
+    def _make_point(self, xy):
+        """ Translate an (x, y) tuple based on image orientation
+            and convert to image coordinates
+
+        Arguments:
+            xy: tuple of (x, y) coordinates for point to translate
+                in format used by Photos (percent of height/width)
+        
+        Returns:
+            (x, y) tuple of translated coordinates in pixels in PIL format/reference frame
+        """
+        # Reference: https://github.com/neilpa/phace/blob/7594776480505d0c389688a42099c94ac5d34f3f/cmd/phace/draw.go#L79-L94
+
+        orientation = self.photo.orientation
+        x, y = xy
+        dx = self.photo.width
+        dy = self.photo.height
+        if orientation in [1, 2]:
+            y = 1.0 - y
+        elif orientation in [3, 4]:
+            x = 1.0 - x
+        elif orientation in [5, 6]:
+            x, y = 1.0 - y, 1.0 - x
+            dx, dy = dy, dx
+        elif orientation in [7, 8]:
+            x, y = y, x
+            dx, dy = dy, dx
+        else:
+            logging.warning(f"Unhandled orientation: {orientation}")
+
+        return (int(x * dx), int(y * dy))
+
+    def _make_point_with_rotation(self, xy):
+        """ Translate an (x, y) tuple based on image orientation and rotation
+            and convert to image coordinates
+
+        Arguments:
+            xy: tuple of (x, y) coordinates for point to translate
+                in format used by Photos (percent of height/width)
+        
+        Returns:
+            (x, y) tuple of translated coordinates in pixels in PIL format/reference frame
+        """
+
+        # convert to image coordinates
+        x, y = self._make_point(xy)
+
+        # rotate about center
+        xmid, ymid = self.center
+        roll, _, _ = self.roll_pitch_yaw()
+        xr, yr = rotate_image_point(x, y, xmid, ymid, roll)
+
+        return (int(xr), int(yr))
+
+    def asdict(self):
+        """ Returns dict representation of class instance """
+        roll, pitch, yaw = self.roll_pitch_yaw()
+        return {
+            "_pk": self._pk,
+            "uuid": self.uuid,
+            "name": self.name,
+            "asset_uuid": self.asset_uuid,
+            "_person_pk": self._person_pk,
+            "center_x": self.center_x,
+            "center_y": self.center_y,
+            "center": self.center,
+            "mouth_x": self.mouth_x,
+            "mouth_y": self.mouth_y,
+            "mouth": self.mouth,
+            "left_eye_x": self.left_eye_x,
+            "left_eye_y": self.left_eye_y,
+            "left_eye": self.left_eye,
+            "right_eye_x": self.right_eye_x,
+            "right_eye_y": self.right_eye_y,
+            "right_eye": self.right_eye,
+            "size": self.size,
+            "face_rect": self.face_rect(),
+            "roll": roll,
+            "pitch": pitch,
+            "yaw": yaw,
+            "quality": self.quality,
+            "source_width": self.source_width,
+            "source_height": self.source_height,
+            "has_smile": self.has_smile,
+            "left_eye_closed": self.left_eye_closed,
+            "right_eye_closed": self.right_eye_closed,
+            "manual": self.manual,
+            "face_type": self.face_type,
+            "age_type": self.age_type,
+            "bald_type": self.bald_type,
+            "eye_makeup_type": self.eye_makeup_type,
+            "eye_state": self.eye_state,
+            "facial_hair_type": self.facial_hair_type,
+            "gender_type": self.gender_type,
+            "glasses_type": self.glasses_type,
+            "hair_color_type": self.hair_color_type,
+            "intrash": self.intrash,
+            "lip_makeup_type": self.lip_makeup_type,
+            "smile_type": self.smile_type,
+        }
+
+    def json(self):
+        """ Return JSON representation of FaceInfo instance """
+        return json.dumps(self.asdict())
+
+    def __str__(self):
+        return f"FaceInfo(uuid={self.uuid}, center_x={self.center_x}, center_y = {self.center_y}, size={self.size}, person={self.name}, asset_uuid={self.asset_uuid})"
+
+    def __repr__(self):
+        return f"FaceInfo(db={self._db}, pk={self._pk})"
+
+    def __eq__(self, other):
+        if not isinstance(other, type(self)):
+            return False
+
+        return all(
+            getattr(self, field) == getattr(other, field) for field in ["_db", "_pk"]
+        )
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+
+def rotate_image_point(x, y, xmid, ymid, angle):
+    """ rotate image point about xm, ym by angle in radians
+
+    Arguments:
+        x: x coordinate of point to rotate    
+        y: y coordinate of point to rotate
+        xmid: x coordinate of center point to rotate about
+        ymid: y coordinate of center point to rotate about
+        angle: angle in radians about which to coordinate, 
+            counter-clockwise is positive
+
+    Returns:
+        tuple of rotated points (xr, yr)
+    """
+    # translate point relative to the mid point
+    x = x - xmid
+    y = y - ymid
+
+    # rotate by angle and translate back
+    # the photo coordinate system is downwards y is positive so
+    # need to adjust the rotation accordingly
+    cos_angle = math.cos(angle)
+    sin_angle = math.sin(angle)
+    xr = x * cos_angle + y * sin_angle + xmid
+    yr = -x * sin_angle + y * cos_angle + ymid
+
+    return (xr, yr)

osxphotos/photoinfo/_photoinfo_exiftool.py

@@ -5,6 +5,7 @@ import os
 
 from ..exiftool import ExifTool, get_exiftool_path
 
+
 @property
 def exiftool(self):
     """ Returns an ExifTool object for the photo
@@ -26,8 +27,9 @@ def exiftool(self):
         except FileNotFoundError:
             # get_exiftool_path raises FileNotFoundError if exiftool not found
             exiftool = None
-            logging.warning(f"exiftool not in path; download and install from https://exiftool.org/")
+            logging.warning(
+                f"exiftool not in path; download and install from https://exiftool.org/"
+            )
 
         self._exiftool = exiftool
         return self._exiftool
-

osxphotos/photoinfo/_photoinfo_export.py

@@ -11,7 +11,6 @@
 
 # TODO: should this be its own PhotoExporter class?
 
-import filecmp
 import glob
 import json
 import logging
@@ -34,10 +33,11 @@ from .._constants import (
 from .._export_db import ExportDBNoOp
 from ..exiftool import ExifTool
 from ..fileutil import FileUtil
-from ..utils import dd_to_dms_str
+from ..utils import dd_to_dms_str, findfiles
 
 ExportResults = namedtuple(
-    "ExportResults", ["exported", "new", "updated", "skipped", "exif_updated"]
+    "ExportResults",
+    ["exported", "new", "updated", "skipped", "exif_updated", "touched"],
 )
 
 
@@ -106,7 +106,7 @@ def _export_photo_uuid_applescript(
     )
 
     dest = pathlib.Path(dest)
-    if not dest.is_dir:
+    if not dest.is_dir():
         raise ValueError(f"dest {dest} must be a directory")
 
     if not original ^ edited:
@@ -215,6 +215,7 @@ def export(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
 ):
     """ export photo 
         dest: must be valid destination path (or exception raised) 
@@ -250,6 +251,7 @@ def export(
         use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords
         when exporting metadata with exiftool or sidecar
         keyword_template: (list of strings); list of template strings that will be rendered as used as keywords
+        description_template: string; optional template string that will be rendered for use as photo description
         returns: list of photos exported
         """
 
@@ -273,6 +275,7 @@ def export(
         use_albums_as_keywords=use_albums_as_keywords,
         use_persons_as_keywords=use_persons_as_keywords,
         keyword_template=keyword_template,
+        description_template=description_template,
     )
 
     return results.exported
@@ -297,10 +300,12 @@ def export2(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
     update=False,
     export_db=None,
     fileutil=FileUtil,
     dry_run=False,
+    touch_file=False,
 ):
     """ export photo, like export but with update and dry_run options
         dest: must be valid destination path or exception raised 
@@ -336,12 +341,14 @@ def export2(
         use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords
         when exporting metadata with exiftool or sidecar
         keyword_template: (list of strings); list of template strings that will be rendered as used as keywords
+        description_template: string; optional template string that will be rendered for use as photo description
         update: (boolean, default=False); if True export will run in update mode, that is, it will
                 not export the photo if the current version already exists in the destination
         export_db: (ExportDB_ABC); instance of a class that conforms to ExportDB_ABC with methods
                 for getting/setting data related to exported files to compare update state
         fileutil: (FileUtilABC); class that conforms to FileUtilABC with various file utilities
         dry_run: (boolean, default=False); set to True to run in "dry run" mode
+        touch_file: (boolean, default=False); if True, sets file's modification time upon photo date
 
         Returns: ExportResults namedtuple with fields: exported, new, updated, skipped 
                     where each field is a list of file paths
@@ -370,6 +377,9 @@ def export2(
     # list of all files skipped because they do not need to be updated (for use with update=True)
     update_skipped_files = []
 
+    # list of all files with utime touched (touch_file = True)
+    touched_files = []
+
     # check edited and raise exception trying to export edited version of
     # photo that hasn't been edited
     if edited and not self.hasadjustments:
@@ -423,11 +433,10 @@ def export2(
     # dest will be file1 (1).jpeg even though file1.jpeg doesn't exist to prevent sidecar collision
     if not update and increment and not overwrite:
         count = 1
-        glob_str = str(dest.parent / f"{dest.stem}*")
-        dest_files = glob.glob(glob_str)
-        dest_files = [pathlib.Path(f).stem for f in dest_files]
+        dest_files = findfiles(f"{dest.stem}*", str(dest.parent))
+        dest_files = [pathlib.Path(f).stem.lower() for f in dest_files]
         dest_new = dest.stem
-        while dest_new in dest_files:
+        while dest_new.lower() in dest_files:
             dest_new = f"{dest.stem} ({count})"
             count += 1
         dest = dest.parent / f"{dest_new}{dest.suffix}"
@@ -478,7 +487,7 @@ def export2(
         if update and dest.exists():
             # destination exists, check to see if destination is the right UUID
             dest_uuid = export_db.get_uuid_for_file(dest)
-            if dest_uuid is None and filecmp.cmp(src, dest):
+            if dest_uuid is None and fileutil.cmp(src, dest):
                 # might be exporting into a pre-ExportDB folder or the DB got deleted
                 logging.debug(
                     f"Found matching file with blank uuid: {self.uuid}, {dest}"
@@ -510,7 +519,7 @@ def export2(
                         dest = pathlib.Path(file_)
                         found_match = True
                         break
-                    elif dest_uuid is None and filecmp.cmp(src, file_):
+                    elif dest_uuid is None and fileutil.cmp(src, file_):
                         # files match, update the UUID
                         logging.debug(
                             f"Found matching file with blank uuid: {self.uuid}, {file_}"
@@ -554,12 +563,14 @@ def export2(
             no_xattr,
             export_as_hardlink,
             exiftool,
+            touch_file,
             fileutil,
         )
         exported_files = results.exported
         update_new_files = results.new
         update_updated_files = results.updated
         update_skipped_files = results.skipped
+        touched_files = results.touched
 
         # copy live photo associated .mov if requested
         if live_photo and self.live_photo:
@@ -579,12 +590,14 @@ def export2(
                     no_xattr,
                     export_as_hardlink,
                     exiftool,
+                    touch_file,
                     fileutil,
                 )
                 exported_files.extend(results.exported)
                 update_new_files.extend(results.new)
                 update_updated_files.extend(results.updated)
                 update_skipped_files.extend(results.skipped)
+                touched_files.extend(results.touched)
             else:
                 logging.debug(f"Skipping missing live movie for {filename}")
 
@@ -604,17 +617,19 @@ def export2(
                     no_xattr,
                     export_as_hardlink,
                     exiftool,
+                    touch_file,
                     fileutil,
                 )
                 exported_files.extend(results.exported)
                 update_new_files.extend(results.new)
                 update_updated_files.extend(results.updated)
                 update_skipped_files.extend(results.skipped)
+                touched_files.extend(results.touched)
             else:
                 logging.debug(f"Skipping missing RAW photo for {filename}")
     else:
         # use_photo_export
-        exported = None
+        exported = []
         # export live_photo .mov file?
         live_photo = True if live_photo and self.live_photo else False
         if edited:
@@ -624,7 +639,7 @@ def export2(
                 filestem = dest.stem
             else:
                 # didn't get passed a filename, add _edited
-                filestem = f"{dest.stem}_edited"
+                filestem = f"{dest.stem}{edited_identifier}"
                 dest = dest.parent / f"{filestem}.jpeg"
 
             exported = _export_photo_uuid_applescript(
@@ -653,23 +668,29 @@ def export2(
                 dry_run=dry_run,
             )
 
-        if exported is not None:
+        if exported:
+            if touch_file:
+                for exported_file in exported:
+                    touched_files.append(exported_file)
+                    ts = int(self.date.timestamp())
+                    fileutil.utime(exported_file, (ts, ts))
             exported_files.extend(exported)
+            if update:
+                update_new_files.extend(exported)
+
         else:
             logging.warning(
                 f"Error exporting photo {self.uuid} to {dest} with use_photos_export"
             )
 
-    # export metadata
-    info = export_db.get_info_for_uuid(self.uuid)
-
     if sidecar_json:
         logging.debug("writing exiftool_json_sidecar")
-        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.json")
+        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}{dest.suffix}.json")
         sidecar_str = self._exiftool_json_sidecar(
             use_albums_as_keywords=use_albums_as_keywords,
             use_persons_as_keywords=use_persons_as_keywords,
             keyword_template=keyword_template,
+            description_template=description_template,
         )
         if not dry_run:
             try:
@@ -680,11 +701,13 @@ def export2(
 
     if sidecar_xmp:
         logging.debug("writing xmp_sidecar")
-        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.xmp")
+        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}{dest.suffix}.xmp")
         sidecar_str = self._xmp_sidecar(
             use_albums_as_keywords=use_albums_as_keywords,
             use_persons_as_keywords=use_persons_as_keywords,
             keyword_template=keyword_template,
+            description_template=description_template,
+            extension=dest.suffix[1:] if dest.suffix else None,
         )
         if not dry_run:
             try:
@@ -712,6 +735,7 @@ def export2(
                         use_albums_as_keywords=use_albums_as_keywords,
                         use_persons_as_keywords=use_persons_as_keywords,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                     )
                 )[0]
                 if old_data != current_data:
@@ -727,6 +751,7 @@ def export2(
                         use_albums_as_keywords=use_albums_as_keywords,
                         use_persons_as_keywords=use_persons_as_keywords,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                     )
                 export_db.set_exifdata_for_file(
                     exported_file,
@@ -734,6 +759,7 @@ def export2(
                         use_albums_as_keywords=use_albums_as_keywords,
                         use_persons_as_keywords=use_persons_as_keywords,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                     ),
                 )
                 export_db.set_stat_exif_for_file(
@@ -749,13 +775,16 @@ def export2(
                     use_albums_as_keywords=use_albums_as_keywords,
                     use_persons_as_keywords=use_persons_as_keywords,
                     keyword_template=keyword_template,
+                    description_template=description_template,
                 )
+
             export_db.set_exifdata_for_file(
                 exported_file,
                 self._exiftool_json_sidecar(
                     use_albums_as_keywords=use_albums_as_keywords,
                     use_persons_as_keywords=use_persons_as_keywords,
                     keyword_template=keyword_template,
+                    description_template=description_template,
                 ),
             )
             export_db.set_stat_exif_for_file(
@@ -763,13 +792,23 @@ def export2(
             )
             exif_files_updated.append(exported_file)
 
-    return ExportResults(
+    if touch_file:
+        for exif_file in exif_files_updated:
+            touched_files.append(exif_file)
+            ts = int(self.date.timestamp())
+            fileutil.utime(exif_file, (ts, ts))
+
+    touched_files = list(set(touched_files))
+
+    results = ExportResults(
         exported_files,
         update_new_files,
         update_updated_files,
         update_skipped_files,
         exif_files_updated,
+        touched_files,
     )
+    return results
 
 
 def _export_photo(
@@ -782,11 +821,12 @@ def _export_photo(
     no_xattr,
     export_as_hardlink,
     exiftool,
+    touch_file,
     fileutil=FileUtil,
 ):
     """ Helper function for export()
         Does the actual copy or hardlink taking the appropriate 
-        action depending on update, overwrite
+        action depending on update, overwrite, export_as_hardlink
         Assumes destination is the right destination (e.g. UUID matches)
         sets UUID and JSON info foo exported file using set_uuid_for_file, set_inf_for_uuido
     
@@ -799,6 +839,7 @@ def _export_photo(
         no_xattr: don't copy extended attributes
         export_as_hardlink: bool
         exiftool: bool
+        touch_file: bool
         fileutil: FileUtil class that conforms to fileutil.FileUtilABC
 
     Returns:
@@ -809,143 +850,99 @@ def _export_photo(
     update_updated_files = []
     update_new_files = []
     update_skipped_files = []
+    touched_files = []
 
     dest_str = str(dest)
     dest_exists = dest.exists()
     if export_as_hardlink:
-        # use hardlink instead of copy
-        if not update:
-            # not update, do the the hardlink
-            if overwrite and dest.exists():
-                # need to remove the destination first
-                # dest.unlink()
-                fileutil.unlink(dest)
-            logging.debug(f"Not update: export_as_hardlink linking file {src} {dest}")
-            fileutil.hardlink(src, dest)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-            exported_files.append(dest_str)
-        elif dest_exists and dest.samefile(src):
-            # update, hardlink and it already points to the right file, do nothing
-            logging.debug(
-                f"Update: skipping samefile with export_as_hardlink {src} {dest}"
-            )
-            update_skipped_files.append(dest_str)
-        elif dest_exists:
-            # update, not the same file (e.g. user may not have used export_as_hardlink last time it was run
-            logging.debug(
-                f"Update: removing existing file prior to export_as_hardlink {src} {dest}"
-            )
-            # dest.unlink()
-            fileutil.unlink(dest)
-            fileutil.hardlink(src, dest)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-            update_updated_files.append(dest_str)
-            exported_files.append(dest_str)
-        else:
-            # update, hardlink, destination doesn't exist (new file)
-            logging.debug(
-                f"Update: exporting new file with export_as_hardlink {src} {dest}"
-            )
-            fileutil.hardlink(src, dest)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-            exported_files.append(dest_str)
-            update_new_files.append(dest_str)
+        op_desc = "export_as_hardlink"
     else:
-        if not update:
-            # not update, do the the copy
-            if overwrite and dest.exists():
-                # need to remove the destination first
-                # dest.unlink()
-                fileutil.unlink(dest)
-            logging.debug(f"Not update: copying file {src} {dest}")
-            fileutil.copy(src, dest_str, norsrc=no_xattr)
-            exported_files.append(dest_str)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-        # elif dest_exists and not exiftool and cmp_file(dest_str, export_db.get_stat_orig_for_file(dest_str)):
-        elif (
-            dest_exists
-            and not exiftool
-            and filecmp.cmp(src, dest)
-            and not dest.samefile(src)
-        ):
-            # destination exists but is identical
-            logging.debug(f"Update: skipping identifical original files {src} {dest}")
-            # call set_stat because code can reach this spot if no export DB but exporting a RAW or live photo
-            # potentially re-writes the data in the database but ensures database is complete
-            export_db.set_stat_orig_for_file(dest_str, fileutil.file_sig(dest_str))
-            update_skipped_files.append(dest_str)
-        elif (
-            dest_exists
-            and exiftool
-            and fileutil.cmp_sig(dest_str, export_db.get_stat_exif_for_file(dest_str))
-            and not dest.samefile(src)
-        ):
-            # destination exists but is identical
-            logging.debug(f"Update: skipping identifical exiftool files {src} {dest}")
-            update_skipped_files.append(dest_str)
-        elif dest_exists:
-            # destination exists but is different or is a hardlink
-            logging.debug(f"Update: removing existing file prior to copy {src} {dest}")
-            stat_src = os.stat(src)
-            stat_dest = os.stat(dest)
-            # dest.unlink()
-            fileutil.unlink(dest)
-            fileutil.copy(src, dest_str, norsrc=no_xattr)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-            exported_files.append(dest_str)
-            update_updated_files.append(dest_str)
-        else:
-            # destination doesn't exist, copy the file
-            logging.debug(f"Update: copying new file {src} {dest}")
-            fileutil.copy(src, dest_str, norsrc=no_xattr)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-            exported_files.append(dest_str)
+        op_desc = "export_by_copying"
+
+    if not update:
+        # not update, export the file
+        logging.debug(f"Exporting file with {op_desc} {src} {dest}")
+        exported_files.append(dest_str)
+        if touch_file:
+            sig = fileutil.file_sig(src)
+            sig = (sig[0], sig[1], int(self.date.timestamp()))
+            if not fileutil.cmp_file_sig(src, sig):
+                touched_files.append(dest_str)
+    else:  # updating
+        if not dest_exists:
+            # update, destination doesn't exist (new file)
+            logging.debug(f"Update: exporting new file with {op_desc} {src} {dest}")
             update_new_files.append(dest_str)
+            if touch_file:
+                touched_files.append(dest_str)
+        else:
+            # update, destination exists, but we might not need to replace it...
+            if exiftool:
+                sig_exif = export_db.get_stat_exif_for_file(dest_str)
+                cmp_orig = fileutil.cmp_file_sig(dest_str, sig_exif)
+                sig_exif = (sig_exif[0], sig_exif[1], int(self.date.timestamp()))
+                cmp_touch = fileutil.cmp_file_sig(dest_str, sig_exif)
+            else:
+                cmp_orig = fileutil.cmp(src, dest)
+                cmp_touch = fileutil.cmp(src, dest, mtime1=int(self.date.timestamp()))
+
+            sig_cmp = cmp_touch if touch_file else cmp_orig
+
+            if (export_as_hardlink and dest.samefile(src)) or (
+                not export_as_hardlink and not dest.samefile(src) and sig_cmp
+            ):
+                # destination exists and signatures match, skip it
+                update_skipped_files.append(dest_str)
+            else:
+                # destination exists but signature is different
+                if touch_file and cmp_orig and not cmp_touch:
+                    # destination exists, signature matches original but does not match expected touch time
+                    # skip exporting but update touch time
+                    update_skipped_files.append(dest_str)
+                    touched_files.append(dest_str)
+                elif not touch_file and cmp_touch and not cmp_orig:
+                    # destination exists, signature matches expected touch but not original
+                    # user likely exported with touch_file and is now exporting without touch_file
+                    # don't update the file because it's same but leave touch time
+                    update_skipped_files.append(dest_str)
+                else:
+                    # destination exists but is different
+                    update_updated_files.append(dest_str)
+                    if touch_file:
+                        touched_files.append(dest_str)
+
+    if not update_skipped_files:
+        if dest_exists and (update or overwrite):
+            # need to remove the destination first
+            logging.debug(
+                f"Update: removing existing file prior to {op_desc} {src} {dest}"
+            )
+            fileutil.unlink(dest)
+        if export_as_hardlink:
+            fileutil.hardlink(src, dest)
+        else:
+            fileutil.copy(src, dest_str, norsrc=no_xattr)
+
+        export_db.set_data(
+            dest_str,
+            self.uuid,
+            fileutil.file_sig(dest_str),
+            (None, None, None),
+            self.json(),
+            None,
+        )
+
+    if touched_files:
+        ts = int(self.date.timestamp())
+        fileutil.utime(dest, (ts, ts))
 
     return ExportResults(
-        exported_files, update_new_files, update_updated_files, update_skipped_files, []
+        exported_files + update_new_files + update_updated_files,
+        update_new_files,
+        update_updated_files,
+        update_skipped_files,
+        [],
+        touched_files,
     )
 
 
@@ -955,6 +952,7 @@ def _write_exif_data(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
 ):
     """ write exif data to image file at filepath
     filepath: full path to the image file """
@@ -966,6 +964,7 @@ def _write_exif_data(
             use_albums_as_keywords=use_albums_as_keywords,
             use_persons_as_keywords=use_persons_as_keywords,
             keyword_template=keyword_template,
+            description_template=description_template,
         )
     )[0]
     for exiftag, val in exif_info.items():
@@ -984,6 +983,7 @@ def _exiftool_json_sidecar(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
 ):
     """ return json string of EXIF details in exiftool sidecar format
         Does not include all the EXIF fields as those are likely already in the image
@@ -1009,7 +1009,13 @@ def _exiftool_json_sidecar(
     exif = {}
     exif["_CreatedBy"] = "osxphotos, https://github.com/RhetTbull/osxphotos"
 
-    if self.description:
+    if description_template is not None:
+        description = self.render_template(
+            description_template, expand_inplace=True, inplace_sep=", "
+        )[0]
+        exif["EXIF:ImageDescription"] = description
+        exif["XMP:Description"] = description
+    elif self.description:
         exif["EXIF:ImageDescription"] = self.description
         exif["XMP:Description"] = self.description
 
@@ -1082,7 +1088,6 @@ def _exiftool_json_sidecar(
         lat_str, lon_str = dd_to_dms_str(lat, lon)
         exif["EXIF:GPSLatitude"] = lat_str
         exif["EXIF:GPSLongitude"] = lon_str
-        exif["Composite:GPSPosition"] = f"{lat_str}, {lon_str}"
         lat_ref = "North" if lat >= 0 else "South"
         lon_ref = "East" if lon >= 0 else "West"
         exif["EXIF:GPSLatitudeRef"] = lat_ref
@@ -1112,16 +1117,28 @@ def _xmp_sidecar(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
+    extension=None,
 ):
     """ returns string for XMP sidecar 
         use_albums_as_keywords: treat album names as keywords
         use_persons_as_keywords: treat person names as keywords
-        keyword_template: (list of strings); list of template strings to render as keywords """
-
-    # TODO: add additional fields to XMP file?
+        keyword_template: (list of strings); list of template strings to render as keywords 
+        description_template: string; optional template string that will be rendered for use as photo description """
 
     xmp_template = Template(filename=os.path.join(_TEMPLATE_DIR, _XMP_TEMPLATE_NAME))
 
+    if extension is None:
+        extension = pathlib.Path(self.original_filename)
+        extension = extension.suffix[1:] if extension.suffix else None
+
+    if description_template is not None:
+        description = self.render_template(
+            description_template, expand_inplace=True, inplace_sep=", "
+        )[0]
+    else:
+        description = self.description if self.description is not None else ""
+
     keyword_list = []
     if self.keywords:
         keyword_list.extend(self.keywords)
@@ -1178,7 +1195,12 @@ def _xmp_sidecar(
         subject_list = list(self.keywords) + person_list
 
     xmp_str = xmp_template.render(
-        photo=self, keywords=keyword_list, persons=person_list, subjects=subject_list
+        photo=self,
+        description=description,
+        keywords=keyword_list,
+        persons=person_list,
+        subjects=subject_list,
+        extension=extension,
     )
 
     # remove extra lines that mako inserts from template

osxphotos/photoinfo/photoinfo.py

@@ -5,16 +5,12 @@ PhotosDB.photos() returns a list of PhotoInfo objects
 """
 
 import dataclasses
-import glob
 import json
 import logging
 import os
 import os.path
 import pathlib
-import subprocess
-import sys
 from datetime import timedelta, timezone
-from pprint import pformat
 
 import yaml
 
@@ -25,10 +21,12 @@ from .._constants import (
     _PHOTOS_4_ROOT_FOLDER,
     _PHOTOS_4_VERSION,
     _PHOTOS_5_ALBUM_KIND,
+    _PHOTOS_5_IMPORT_SESSION_ALBUM_KIND,
     _PHOTOS_5_SHARED_ALBUM_KIND,
     _PHOTOS_5_SHARED_PHOTO_PATH,
 )
-from ..albuminfo import AlbumInfo
+from ..albuminfo import AlbumInfo, ImportInfo
+from ..personinfo import FaceInfo, PersonInfo
 from ..phototemplate import PhotoTemplate
 from ..placeinfo import PlaceInfo4, PlaceInfo5
 from ..utils import _debug, _get_resource_loc, findfiles, get_preferred_uti_extension
@@ -86,11 +84,7 @@ class PhotoInfo:
     @property
     def date(self):
         """ image creation date as timezone aware datetime object """
-        imagedate = self._info["imageDate"]
-        seconds = self._info["imageTimeZoneOffsetSeconds"] or 0
-        delta = timedelta(seconds=seconds)
-        tz = timezone(delta)
-        return imagedate.astimezone(tz=tz)
+        return self._info["imageDate"]
 
     @property
     def date_modified(self):
@@ -339,7 +333,32 @@ class PhotoInfo:
     @property
     def persons(self):
         """ list of persons in picture """
-        return self._info["persons"]
+        return [self._db._dbpersons_pk[pk]["fullname"] for pk in self._info["persons"]]
+
+    @property
+    def person_info(self):
+        """ list of PersonInfo objects for person in picture """
+        try:
+            return self._personinfo
+        except AttributeError:
+            self._personinfo = [
+                PersonInfo(db=self._db, pk=pk) for pk in self._info["persons"]
+            ]
+            return self._personinfo
+
+    @property
+    def face_info(self):
+        """ list of FaceInfo objects for faces in picture """
+        try:
+            return self._faceinfo
+        except AttributeError:
+            try:
+                faces = self._db._db_faceinfo_uuid[self._uuid]
+                self._faceinfo = [FaceInfo(db=self._db, pk=pk) for pk in faces]
+            except KeyError:
+                # no faces
+                self._faceinfo = []
+            return self._faceinfo
 
     @property
     def albums(self):
@@ -365,6 +384,19 @@ class PhotoInfo:
             ]
             return self._album_info
 
+    @property
+    def import_info(self):
+        """ ImportInfo object representing import session for the photo or None if no import session """
+        try:
+            return self._import_info
+        except AttributeError:
+            self._import_info = (
+                ImportInfo(db=self._db, uuid=self._info["import_uuid"])
+                if self._info["import_uuid"] is not None
+                else None
+            )
+            return self._import_info
+
     @property
     def keywords(self):
         """ list of keywords for picture """
@@ -642,7 +674,49 @@ class PhotoInfo:
             otherwise returns False """
         return self._info["raw_is_original"]
 
-    def render_template(self, template_str, none_str="_", path_sep=None):
+    @property
+    def height(self):
+        """ returns height of the current photo version in pixels """
+        return self._info["height"]
+
+    @property
+    def width(self):
+        """ returns width of the current photo version in pixels """
+        return self._info["width"]
+
+    @property
+    def orientation(self):
+        """ returns EXIF orientation of the current photo version as int """
+        return self._info["orientation"]
+
+    @property
+    def original_height(self):
+        """ returns height of the original photo version in pixels """
+        return self._info["original_height"]
+
+    @property
+    def original_width(self):
+        """ returns width of the original photo version in pixels """
+        return self._info["original_width"]
+
+    @property
+    def original_orientation(self):
+        """ returns EXIF orientation of the original photo version as int """
+        return self._info["original_orientation"]
+
+    @property
+    def original_filesize(self):
+        """ returns filesize of original photo in bytes as int """
+        return self._info["original_filesize"]
+
+    def render_template(
+        self,
+        template_str,
+        none_str="_",
+        path_sep=None,
+        expand_inplace=False,
+        inplace_sep=None,
+    ):
         """Renders a template string for PhotoInfo instance using PhotoTemplate
 
         Args:
@@ -650,12 +724,22 @@ class PhotoInfo:
             none_str: a str to use if template field renders to None, default is "_".
             path_sep: a single character str to use as path separator when joining 
                 fields like folder_album; if not provided, defaults to os.path.sep
+            expand_inplace: expand multi-valued substitutions in-place as a single string 
+                instead of returning individual strings
+            inplace_sep: optional string to use as separator between multi-valued keywords
+                with expand_inplace; default is ','
         
         Returns:
             ([rendered_strings], [unmatched]): tuple of list of rendered strings and list of unmatched template values
         """
         template = PhotoTemplate(self)
-        return template.render(template_str, none_str, path_sep)
+        return template.render(
+            template_str,
+            none_str=none_str,
+            path_sep=path_sep,
+            expand_inplace=expand_inplace,
+            inplace_sep=inplace_sep,
+        )
 
     @property
     def _longitude(self):
@@ -671,7 +755,7 @@ class PhotoInfo:
         """ Return list of album UUIDs this photo is found in
         
             Filters out albums in the trash and any special album types
-        
+
         Returns: list of album UUIDs 
         """
         if self._db._db_version <= _PHOTOS_4_VERSION:
@@ -754,6 +838,13 @@ class PhotoInfo:
             "exif": exif,
             "score": score,
             "intrash": self.intrash,
+            "height": self.height,
+            "width": self.width,
+            "orientation": self.orientation,
+            "original_height": self.original_height,
+            "original_width": self.original_width,
+            "original_orientation": self.original_orientation,
+            "original_filesize": self.original_filesize,
         }
         return yaml.dump(info, sort_keys=False)
 
@@ -814,6 +905,13 @@ class PhotoInfo:
             "exif": exif,
             "score": score,
             "intrash": self.intrash,
+            "height": self.height,
+            "width": self.width,
+            "orientation": self.orientation,
+            "original_height": self.original_height,
+            "original_width": self.original_width,
+            "original_orientation": self.original_orientation,
+            "original_filesize": self.original_filesize,
         }
         return json.dumps(pic)
 

osxphotos/photosdb/__init__.py

@@ -4,3 +4,4 @@ Processes a Photos.app library database to extract information about photos
 """
 
 from .photosdb import PhotosDB
+from .photosdb_utils import get_db_version, get_db_model_version, get_model_version

osxphotos/photosdb/_photosdb_process_exif.py

@@ -3,9 +3,9 @@
 
 import logging
 
-from .._constants import _PHOTOS_4_VERSION
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION
 from ..utils import _db_is_locked, _debug, _open_sql_file
-
+from .photosdb_utils import get_db_version
 
 def _process_exifinfo(self):
     """ load the exif data from the database 
@@ -34,15 +34,17 @@ def _process_exifinfo_5(photosdb):
         photosdb: PhotosDB instance """
 
     db = photosdb._tmp_db
-    
+
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+        
     (conn, cursor) = _open_sql_file(db)
 
     result = conn.execute(
-        """
-        SELECT ZGENERICASSET.ZUUID, ZEXTENDEDATTRIBUTES.*
-        FROM ZGENERICASSET
+        f"""
+        SELECT {asset_table}.ZUUID, ZEXTENDEDATTRIBUTES.*
+        FROM {asset_table}
         JOIN ZEXTENDEDATTRIBUTES
-        ON ZEXTENDEDATTRIBUTES.ZASSET = ZGENERICASSET.Z_PK
+        ON ZEXTENDEDATTRIBUTES.ZASSET = {asset_table}.Z_PK
         """
     )
 
@@ -54,3 +56,5 @@ def _process_exifinfo_5(photosdb):
         if uuid in photosdb._db_exifinfo_uuid:
             logging.warning(f"duplicate exifinfo record found for uuid {uuid}")
         photosdb._db_exifinfo_uuid[uuid] = record
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_faceinfo.py

@@ -0,0 +1,331 @@
+""" Methods for PhotosDB to add Photos face info 
+"""
+
+import logging
+
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION
+from ..utils import _open_sql_file, normalize_unicode
+from .photosdb_utils import get_db_version
+
+
+"""
+    This module should be imported in the class defintion of PhotosDB in photosdb.py
+    Do not import this module directly
+    This module adds the following method to PhotosDB:
+        _process_faceinfo: process photo face info 
+
+    The following data structures are added to PhotosDB
+        self._db_faceinfo_pk: {pk: {faceinfo}}
+        self._db_faceinfo_uuid: {photo uuid: [face pk]}
+        self._db_faceinfo_person: {person_pk: [face_pk]}
+"""
+
+
+def _process_faceinfo(self):
+    """ Process face information
+    """
+
+    self._db_faceinfo_pk = {}
+    self._db_faceinfo_uuid = {}
+    self._db_faceinfo_person = {}
+
+    if self._db_version <= _PHOTOS_4_VERSION:
+        _process_faceinfo_4(self)
+    else:
+        _process_faceinfo_5(self)
+
+
+def _process_faceinfo_4(photosdb):
+    """ Process face information for Photos 4 databases
+
+    Args:
+        photosdb: an OSXPhotosDB instance
+    """
+    db = photosdb._tmp_db
+
+    (conn, cursor) = _open_sql_file(db)
+
+    result = cursor.execute(
+        """
+        SELECT
+        RKFace.modelId,
+        RKVersion.uuid, 
+        RKFace.uuid,
+        RKPerson.name,
+        RKFace.isInTrash,
+        RKFace.personId,
+        RKFace.imageModelId, 
+        RKFace.sourceWidth,
+        RKFace.sourceHeight, 
+        RKFace.centerX,
+        RKFace.centerY,
+        RKFace.size,
+        RKFace.leftEyeX, 
+        RKFace.leftEyeY,
+        RKFace.rightEyeX, 
+        RKFace.rightEyeY, 
+        RKFace.mouthX,
+        RKFace.mouthY, 
+        RKFace.hidden, 
+        RKFace.manual, 
+        RKFace.hasSmile, 
+        RKFace.isLeftEyeClosed, 
+        RKFace.isRightEyeClosed, 
+        RKFace.poseRoll, 
+        RKFace.poseYaw, 
+        RKFace.posePitch, 
+        RKFace.faceType,
+        RKFace.qualityMeasure
+        FROM
+        RKFace
+        JOIN RKPerson on RKPerson.modelId = RKFace.personId
+        JOIN RKVersion on RKVersion.modelId = RKFace.imageModelId
+        """
+    )
+
+    # 0     RKFace.modelId,
+    # 1     RKVersion.uuid,
+    # 2     RKFace.uuid,
+    # 3     RKPerson.name,
+    # 4     RKFace.isInTrash,
+    # 5     RKFace.personId,
+    # 6     RKFace.imageModelId,
+    # 7     RKFace.sourceWidth,
+    # 8     RKFace.sourceHeight,
+    # 9     RKFace.centerX,
+    # 10    RKFace.centerY,
+    # 11    RKFace.size,
+    # 12    RKFace.leftEyeX,
+    # 13    RKFace.leftEyeY,
+    # 14    RKFace.rightEyeX,
+    # 15    RKFace.rightEyeY,
+    # 16    RKFace.mouthX,
+    # 17    RKFace.mouthY,
+    # 18    RKFace.hidden,
+    # 19    RKFace.manual,
+    # 20    RKFace.hasSmile,
+    # 21    RKFace.isLeftEyeClosed,
+    # 22    RKFace.isRightEyeClosed,
+    # 23    RKFace.poseRoll,
+    # 24    RKFace.poseYaw,
+    # 25    RKFace.posePitch,
+    # 26    RKFace.faceType,
+    # 27    RKFace.qualityMeasure
+
+    for row in result:
+        modelid = row[0]
+        asset_uuid = row[1]
+        person_id = row[5]
+        face = {}
+        face["pk"] = modelid
+        face["asset_uuid"] = asset_uuid
+        face["uuid"] = row[2]
+        face["person"] = person_id
+        face["fullname"] = normalize_unicode(row[3])
+        face["sourcewidth"] = row[7]
+        face["sourceheight"] = row[8]
+        face["centerx"] = row[9]
+        face["centery"] = row[10]
+        face["size"] = row[11]
+        face["lefteyex"] = row[12]
+        face["lefteyey"] = row[13]
+        face["righteyex"] = row[14]
+        face["righteyey"] = row[15]
+        face["mouthx"] = row[16]
+        face["mouthy"] = row[17]
+        face["hidden"] = row[18]
+        face["manual"] = row[19]
+        face["has_smile"] = row[20]
+        face["left_eye_closed"] = row[21]
+        face["right_eye_closed"] = row[22]
+        face["roll"] = row[23]
+        face["yaw"] = row[24]
+        face["pitch"] = row[25]
+        face["facetype"] = row[26]
+        face["quality"] = row[27]
+
+        # Photos 5 only
+        face["agetype"] = None
+        face["baldtype"] = None
+        face["eyemakeuptype"] = None
+        face["eyestate"] = None
+        face["facialhairtype"] = None
+        face["gendertype"] = None
+        face["glassestype"] = None
+        face["haircolortype"] = None
+        face["intrash"] = None
+        face["lipmakeuptype"] = None
+        face["smiletype"] = None
+
+        photosdb._db_faceinfo_pk[modelid] = face
+
+        try:
+            photosdb._db_faceinfo_uuid[asset_uuid].append(modelid)
+        except KeyError:
+            photosdb._db_faceinfo_uuid[asset_uuid] = [modelid]
+
+        try:
+            photosdb._db_faceinfo_person[person_id].append(modelid)
+        except KeyError:
+            photosdb._db_faceinfo_person[person_id] = [modelid]
+
+    conn.close()
+
+
+def _process_faceinfo_5(photosdb):
+    """ Process face information for Photos 5 databases
+
+    Args:
+        photosdb: an OSXPhotosDB instance
+    """
+
+    db = photosdb._tmp_db
+
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+
+    (conn, cursor) = _open_sql_file(db)
+
+    result = cursor.execute(
+        f""" 
+        SELECT
+        ZDETECTEDFACE.Z_PK,
+        {asset_table}.ZUUID,
+        ZDETECTEDFACE.ZUUID,
+        ZDETECTEDFACE.ZPERSON,
+        ZPERSON.ZFULLNAME,
+        ZDETECTEDFACE.ZAGETYPE,
+        ZDETECTEDFACE.ZBALDTYPE,
+        ZDETECTEDFACE.ZEYEMAKEUPTYPE,
+        ZDETECTEDFACE.ZEYESSTATE,
+        ZDETECTEDFACE.ZFACIALHAIRTYPE,
+        ZDETECTEDFACE.ZGENDERTYPE,
+        ZDETECTEDFACE.ZGLASSESTYPE,
+        ZDETECTEDFACE.ZHAIRCOLORTYPE,
+        ZDETECTEDFACE.ZHASSMILE,
+        ZDETECTEDFACE.ZHIDDEN,
+        ZDETECTEDFACE.ZISINTRASH,
+        ZDETECTEDFACE.ZISLEFTEYECLOSED,
+        ZDETECTEDFACE.ZISRIGHTEYECLOSED,
+        ZDETECTEDFACE.ZLIPMAKEUPTYPE,
+        ZDETECTEDFACE.ZMANUAL,
+        ZDETECTEDFACE.ZQUALITYMEASURE,
+        ZDETECTEDFACE.ZSMILETYPE,
+        ZDETECTEDFACE.ZSOURCEHEIGHT,
+        ZDETECTEDFACE.ZSOURCEWIDTH,
+        ZDETECTEDFACE.ZBLURSCORE,
+        ZDETECTEDFACE.ZCENTERX,
+        ZDETECTEDFACE.ZCENTERY,
+        ZDETECTEDFACE.ZLEFTEYEX,
+        ZDETECTEDFACE.ZLEFTEYEY,
+        ZDETECTEDFACE.ZMOUTHX,
+        ZDETECTEDFACE.ZMOUTHY,
+        ZDETECTEDFACE.ZPOSEYAW,
+        ZDETECTEDFACE.ZQUALITY,
+        ZDETECTEDFACE.ZRIGHTEYEX,
+        ZDETECTEDFACE.ZRIGHTEYEY,
+        ZDETECTEDFACE.ZROLL,
+        ZDETECTEDFACE.ZSIZE,
+        ZDETECTEDFACE.ZYAW,
+        ZDETECTEDFACE.ZMASTERIDENTIFIER
+        FROM ZDETECTEDFACE
+        JOIN {asset_table} ON {asset_table}.Z_PK = ZDETECTEDFACE.ZASSET
+        JOIN ZPERSON ON ZPERSON.Z_PK = ZDETECTEDFACE.ZPERSON;
+        """
+    )
+
+    # 0    ZDETECTEDFACE.Z_PK
+    # 1    ZGENERICASSET.ZUUID,
+    # 2    ZDETECTEDFACE.ZUUID,
+    # 3    ZDETECTEDFACE.ZPERSON,
+    # 4    ZPERSON.ZFULLNAME,
+    # 5    ZDETECTEDFACE.ZAGETYPE,
+    # 6    ZDETECTEDFACE.ZBALDTYPE,
+    # 7    ZDETECTEDFACE.ZEYEMAKEUPTYPE,
+    # 8    ZDETECTEDFACE.ZEYESSTATE,
+    # 9    ZDETECTEDFACE.ZFACIALHAIRTYPE,
+    # 10   ZDETECTEDFACE.ZGENDERTYPE,
+    # 11   ZDETECTEDFACE.ZGLASSESTYPE,
+    # 12   ZDETECTEDFACE.ZHAIRCOLORTYPE,
+    # 13   ZDETECTEDFACE.ZHASSMILE,
+    # 14   ZDETECTEDFACE.ZHIDDEN,
+    # 15   ZDETECTEDFACE.ZISINTRASH,
+    # 16   ZDETECTEDFACE.ZISLEFTEYECLOSED,
+    # 17   ZDETECTEDFACE.ZISRIGHTEYECLOSED,
+    # 18   ZDETECTEDFACE.ZLIPMAKEUPTYPE,
+    # 19   ZDETECTEDFACE.ZMANUAL,
+    # 20   ZDETECTEDFACE.ZQUALITYMEASURE,
+    # 21   ZDETECTEDFACE.ZSMILETYPE,
+    # 22   ZDETECTEDFACE.ZSOURCEHEIGHT,
+    # 23   ZDETECTEDFACE.ZSOURCEWIDTH,
+    # 24   ZDETECTEDFACE.ZBLURSCORE,
+    # 25   ZDETECTEDFACE.ZCENTERX,
+    # 26   ZDETECTEDFACE.ZCENTERY,
+    # 27   ZDETECTEDFACE.ZLEFTEYEX,
+    # 28   ZDETECTEDFACE.ZLEFTEYEY,
+    # 29   ZDETECTEDFACE.ZMOUTHX,
+    # 30   ZDETECTEDFACE.ZMOUTHY,
+    # 31   ZDETECTEDFACE.ZPOSEYAW,
+    # 32   ZDETECTEDFACE.ZQUALITY,
+    # 33   ZDETECTEDFACE.ZRIGHTEYEX,
+    # 34   ZDETECTEDFACE.ZRIGHTEYEY,
+    # 35   ZDETECTEDFACE.ZROLL,
+    # 36   ZDETECTEDFACE.ZSIZE,
+    # 37   ZDETECTEDFACE.ZYAW,
+    # 38   ZDETECTEDFACE.ZMASTERIDENTIFIER
+
+    for row in result:
+        pk = row[0]
+        asset_uuid = row[1]
+        person_pk = row[3]
+        face = {}
+        face["pk"] = pk
+        face["asset_uuid"] = asset_uuid
+        face["uuid"] = row[2]
+        face["person"] = person_pk
+        face["fullname"] = normalize_unicode(row[4])
+        face["agetype"] = row[5]
+        face["baldtype"] = row[6]
+        face["eyemakeuptype"] = row[7]
+        face["eyestate"] = row[8]
+        face["facialhairtype"] = row[9]
+        face["gendertype"] = row[10]
+        face["glassestype"] = row[11]
+        face["haircolortype"] = row[12]
+        face["has_smile"] = row[13]
+        face["hidden"] = row[14]
+        face["intrash"] = row[15]
+        face["left_eye_closed"] = row[16]
+        face["right_eye_closed"] = row[17]
+        face["lipmakeuptype"] = row[18]
+        face["manual"] = row[19]
+        face["smiletype"] = row[21]
+        face["sourceheight"] = row[22]
+        face["sourcewidth"] = row[23]
+        face["facetype"] = None  # Photos 4 only
+        face["centerx"] = row[25]
+        face["centery"] = row[26]
+        face["lefteyex"] = row[27]
+        face["lefteyey"] = row[28]
+        face["mouthx"] = row[29]
+        face["mouthy"] = row[30]
+        face["quality"] = row[32]
+        face["righteyex"] = row[33]
+        face["righteyey"] = row[34]
+        face["roll"] = row[35]
+        face["size"] = row[36]
+        face["yaw"] = row[37]
+        face["pitch"] = 0.0  # not defined in Photos 5
+
+        photosdb._db_faceinfo_pk[pk] = face
+
+        try:
+            photosdb._db_faceinfo_uuid[asset_uuid].append(pk)
+        except KeyError:
+            photosdb._db_faceinfo_uuid[asset_uuid] = [pk]
+
+        try:
+            photosdb._db_faceinfo_person[person_pk].append(pk)
+        except KeyError:
+            photosdb._db_faceinfo_person[person_pk] = [pk]
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_scoreinfo.py

@@ -4,8 +4,9 @@
 
 import logging
 
-from .._constants import _PHOTOS_4_VERSION
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION
 from ..utils import _open_sql_file
+from .photosdb_utils import get_db_version
 
 """
     This module should be imported in the class defintion of PhotosDB in photosdb.py
@@ -45,16 +46,18 @@ def _process_scoreinfo_5(photosdb):
 
     db = photosdb._tmp_db
 
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+
     (conn, cursor) = _open_sql_file(db)
 
     result = cursor.execute(
-        """
+        f"""
         SELECT 
-        ZGENERICASSET.ZUUID,
-        ZGENERICASSET.ZOVERALLAESTHETICSCORE,
-        ZGENERICASSET.ZCURATIONSCORE,
-        ZGENERICASSET.ZPROMOTIONSCORE,
-        ZGENERICASSET.ZHIGHLIGHTVISIBILITYSCORE,
+        {asset_table}.ZUUID,
+        {asset_table}.ZOVERALLAESTHETICSCORE,
+        {asset_table}.ZCURATIONSCORE,
+        {asset_table}.ZPROMOTIONSCORE,
+        {asset_table}.ZHIGHLIGHTVISIBILITYSCORE,
         ZCOMPUTEDASSETATTRIBUTES.ZBEHAVIORALSCORE, 
         ZCOMPUTEDASSETATTRIBUTES.ZFAILURESCORE,
         ZCOMPUTEDASSETATTRIBUTES.ZHARMONIOUSCOLORSCORE,
@@ -78,8 +81,8 @@ def _process_scoreinfo_5(photosdb):
         ZCOMPUTEDASSETATTRIBUTES.ZWELLCHOSENSUBJECTSCORE,
         ZCOMPUTEDASSETATTRIBUTES.ZWELLFRAMEDSUBJECTSCORE,
         ZCOMPUTEDASSETATTRIBUTES.ZWELLTIMEDSHOTSCORE
-        FROM ZGENERICASSET
-        JOIN ZCOMPUTEDASSETATTRIBUTES ON ZCOMPUTEDASSETATTRIBUTES.ZASSET = ZGENERICASSET.Z_PK
+        FROM {asset_table}
+        JOIN ZCOMPUTEDASSETATTRIBUTES ON ZCOMPUTEDASSETATTRIBUTES.ZASSET = {asset_table}.Z_PK
         """
     )
 
@@ -143,3 +146,5 @@ def _process_scoreinfo_5(photosdb):
         scores["well_framed_subject"] = row[26]
         scores["well_timed_shot"] = row[27]
         photosdb._db_scoreinfo_uuid[uuid] = scores
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_searchinfo.py

@@ -10,7 +10,7 @@ import uuid as uuidlib
 from pprint import pformat
 
 from .._constants import _PHOTOS_4_VERSION, SEARCH_CATEGORY_LABEL
-from ..utils import _db_is_locked, _debug, _open_sql_file
+from ..utils import _db_is_locked, _debug, _open_sql_file, normalize_unicode
 
 """
     This module should be imported in the class defintion of PhotosDB in photosdb.py
@@ -112,8 +112,8 @@ def _process_searchinfo(self):
         record["groupid"] = row[3]
         record["category"] = row[4]
         record["owning_groupid"] = row[5]
-        record["content_string"] = row[6].replace("\x00", "")
-        record["normalized_string"] = row[7].replace("\x00", "")
+        record["content_string"] = normalize_unicode(row[6].replace("\x00", ""))
+        record["normalized_string"] = normalize_unicode(row[7].replace("\x00", ""))
         record["lookup_identifier"] = row[8]
 
         try:
@@ -148,6 +148,8 @@ def _process_searchinfo(self):
             + pformat(self._db_searchinfo_labels_normalized)
         )
 
+    conn.close()
+
 
 @property
 def labels(self):

osxphotos/photosdb/photosdb_utils.py

@@ -0,0 +1,84 @@
+""" utility functions used by PhotosDB """
+
+import logging
+import plistlib
+
+from .._constants import (
+    _PHOTOS_5_MODEL_VERSION,
+    _PHOTOS_6_MODEL_VERSION,
+    _TESTED_DB_VERSIONS,
+)
+from ..utils import _open_sql_file
+
+
+def get_db_version(db_file):
+    """ Gets the Photos DB version from LiGlobals table
+
+    Args:
+        db_file: path to photos.db database file containing LiGlobals table
+
+    Returns: version as str
+    """
+
+    version = None
+
+    (conn, c) = _open_sql_file(db_file)
+
+    # get database version
+    c.execute("SELECT value from LiGlobals where LiGlobals.keyPath is 'libraryVersion'")
+    version = c.fetchone()[0]
+    conn.close()
+
+    if version not in _TESTED_DB_VERSIONS:
+        print(
+            f"WARNING: Only tested on database versions [{', '.join(_TESTED_DB_VERSIONS)}]"
+            + f" You have database version={version} which has not been tested"
+        )
+
+    return version
+
+
+def get_model_version(db_file):
+    """ Returns the database model version from Z_METADATA
+    
+    Args:
+        db_file: path to Photos.sqlite database file containing Z_METADATA table
+        
+    Returns: model version as str
+    """
+
+    version = None
+
+    (conn, c) = _open_sql_file(db_file)
+
+    # get database version
+    c.execute("SELECT MAX(Z_VERSION), Z_PLIST FROM Z_METADATA")
+    results = c.fetchone()
+
+    conn.close()
+
+    plist = plistlib.loads(results[1])
+    return plist["PLModelVersion"]
+
+
+def get_db_model_version(db_file):
+    """ Returns Photos version based on model version found in db_file
+    
+    Args:
+        db_file: path to Photos.sqlite file
+    
+    Returns: int of major Photos version number (e.g. 5 or 6).
+    If unknown model version found, logs warning and returns most current Photos version.
+    """
+
+    model_ver = get_model_version(db_file)
+    if _PHOTOS_5_MODEL_VERSION[0] <= model_ver <= _PHOTOS_5_MODEL_VERSION[1]:
+        db_ver = 5
+    elif _PHOTOS_6_MODEL_VERSION[0] <= model_ver <= _PHOTOS_6_MODEL_VERSION[1]:
+        db_ver = 6
+    else:
+        logging.warning(f"Unknown model version: {model_ver}")
+        # cross our fingers and try latest version
+        db_ver = 6
+
+    return db_ver

osxphotos/phototemplate.py

@@ -124,13 +124,24 @@ class PhotoTemplate:
         # gets initialized in get_template_value
         self.today = None
 
-    def render(self, template, none_str="_", path_sep=None):
+    def render(
+        self,
+        template,
+        none_str="_",
+        path_sep=None,
+        expand_inplace=False,
+        inplace_sep=None,
+    ):
         """ Render a filename or directory template 
 
         Args:
             template: str template 
             none_str: str to use default for None values, default is '_' 
             path_sep: optional character to use as path separator, default is os.path.sep
+            expand_inplace: expand multi-valued substitutions in-place as a single string 
+                instead of returning individual strings
+            inplace_sep: optional string to use as separator between multi-valued keywords
+            with expand_inplace; default is ','
 
         Returns:
             ([rendered_strings], [unmatched]): tuple of list of rendered strings and list of unmatched template values
@@ -141,6 +152,9 @@ class PhotoTemplate:
         elif path_sep is not None and len(path_sep) != 1:
             raise ValueError(f"path_sep must be single character: {path_sep}")
 
+        if inplace_sep is None:
+            inplace_sep = ","
+
         # the rendering happens in two phases:
         # phase 1: handle all the single-value template substitutions
         #          results in a single string with all the template fields replaced
@@ -226,13 +240,19 @@ class PhotoTemplate:
             for str_template in rendered_strings:
                 if regex_multi.search(str_template):
                     values = self.get_template_value_multi(field, path_sep)
-                    for val in values:
+                    if expand_inplace:
+                        # instead of returning multiple strings, join values into a single string
+                        val = (
+                            inplace_sep.join(sorted(values))
+                            if values and values[0]
+                            else None
+                        )
 
                         def lookup_template_value_multi(lookup_value, default):
                             """ Closure passed to make_subst_function get_func 
-                                Capture val and field in the closure 
-                                Allows make_subst_function to be re-used w/o modification
-                                default is not used but required so signature matches get_template_value """
+                                    Capture val and field in the closure 
+                                    Allows make_subst_function to be re-used w/o modification
+                                    default is not used but required so signature matches get_template_value """
                             if lookup_value == field:
                                 return val
                             else:
@@ -242,10 +262,33 @@ class PhotoTemplate:
                             self, none_str, get_func=lookup_template_value_multi
                         )
                         new_string = regex_multi.sub(subst, str_template)
-                        new_strings.add(new_string)
 
-                    # update rendered_strings for the next field to process
-                    rendered_strings = new_strings
+                        # update rendered_strings for the next field to process
+                        rendered_strings = {new_string}
+                    else:
+                        # create a new template string for each value
+                        for val in values:
+
+                            def lookup_template_value_multi(lookup_value, default):
+                                """ Closure passed to make_subst_function get_func 
+                                    Capture val and field in the closure 
+                                    Allows make_subst_function to be re-used w/o modification
+                                    default is not used but required so signature matches get_template_value """
+                                if lookup_value == field:
+                                    return val
+                                else:
+                                    raise ValueError(
+                                        f"Unexpected value: {lookup_value}"
+                                    )
+
+                            subst = make_subst_function(
+                                self, none_str, get_func=lookup_template_value_multi
+                            )
+                            new_string = regex_multi.sub(subst, str_template)
+                            new_strings.add(new_string)
+
+                        # update rendered_strings for the next field to process
+                        rendered_strings = new_strings
 
         # find any {fields} that weren't replaced
         unmatched = []
@@ -578,6 +621,9 @@ class PhotoTemplate:
         """ return list of values for a multi-valued template field """
         if field == "album":
             values = self.photo.albums
+            values = [
+                value.replace("/", ":") for value in values
+            ]  # TODO: temp fix for issue #213
         elif field == "keyword":
             values = self.photo.keywords
         elif field == "person":
@@ -595,11 +641,13 @@ class PhotoTemplate:
                 if album.folder_names:
                     # album in folder
                     folder = path_sep.join(album.folder_names)
-                    folder += path_sep + album.title
+                    folder += path_sep + album.title.replace(
+                        "/", ":"
+                    )  # TODO: temp fix for issue #213
                     values.append(folder)
                 else:
                     # album not in folder
-                    values.append(album.title)
+                    values.append(album.title.replace("/", ":"))
         else:
             raise ValueError(f"Unhandleded template value: {field}")
 

osxphotos/placeinfo.py

@@ -11,6 +11,9 @@ from collections import namedtuple  # pylint: disable=syntax-error
 import yaml
 from bpylist import archiver
 
+from ._constants import UNICODE_FORMAT
+from .utils import normalize_unicode
+
 # postal address information, returned by PlaceInfo.address
 PostalAddress = namedtuple(
     "PostalAddress",
@@ -76,12 +79,12 @@ class PLRevGeoLocationInfo:
         geoServiceProvider,
         postalAddress,
     ):
-        self.addressString = addressString
+        self.addressString = normalize_unicode(addressString)
         self.countryCode = countryCode
         self.mapItem = mapItem
         self.isHome = isHome
-        self.compoundNames = compoundNames
-        self.compoundSecondaryNames = compoundSecondaryNames
+        self.compoundNames = normalize_unicode(compoundNames)
+        self.compoundSecondaryNames = normalize_unicode(compoundSecondaryNames)
         self.version = version
         self.geoServiceProvider = geoServiceProvider
         self.postalAddress = postalAddress
@@ -183,7 +186,7 @@ class PLRevGeoMapItemAdditionalPlaceInfo:
 
     def __init__(self, area, name, placeType, dominantOrderType):
         self.area = area
-        self.name = name
+        self.name = normalize_unicode(name)
         self.placeType = placeType
         self.dominantOrderType = dominantOrderType
 
@@ -232,13 +235,13 @@ class CNPostalAddress:
         _subLocality,
     ):
         self._ISOCountryCode = _ISOCountryCode
-        self._city = _city
-        self._country = _country
-        self._postalCode = _postalCode
-        self._state = _state
-        self._street = _street
-        self._subAdministrativeArea = _subAdministrativeArea
-        self._subLocality = _subLocality
+        self._city = normalize_unicode(_city)
+        self._country = normalize_unicode(_country)
+        self._postalCode = normalize_unicode(_postalCode)
+        self._state = normalize_unicode(_state)
+        self._street = normalize_unicode(_street)
+        self._subAdministrativeArea = normalize_unicode(_subAdministrativeArea)
+        self._subLocality = normalize_unicode(_subLocality)
 
     def __eq__(self, other):
         return all(
@@ -414,9 +417,9 @@ class PlaceInfo4(PlaceInfo):
             #   2: type
             #   3: area
             try:
-                places_dict[p[2]].append((p[1], p[3]))
+                places_dict[p[2]].append((normalize_unicode(p[1]), p[3]))
             except KeyError:
-                places_dict[p[2]] = [(p[1], p[3])]
+                places_dict[p[2]] = [(normalize_unicode(p[1]), p[3])]
 
         # build list to populate PlaceNames tuple
         # initialize with empty lists for each field in PlaceNames

osxphotos/templates/xmp_sidecar.mako

@@ -1,5 +1,13 @@
 <!-- Created with osxphotos https://github.com/RhetTbull/osxphotos -->
 
+<%def name="photoshop_sidecar_for_extension(extension)">
+    % if extension is None:
+        <photoshop:SidecarForExtension></photoshop:SidecarForExtension>
+    % else:
+        <photoshop:SidecarForExtension>${extension}</photoshop:SidecarForExtension>
+    % endif
+</%def>
+
 <%def name="dc_description(desc)">
     % if desc is None:
         <dc:description></dc:description>
@@ -71,29 +79,43 @@
     % endif
 </%def>
 
+<%def name="gps_info(latitude, longitude)">
+    % if latitude is not None and longitude is not None:
+        <exif:GPSLongitudeRef>${"E" if longitude >= 0 else "W"}</exif:GPSLongitudeRef>
+        <exif:GPSLongitude>${abs(longitude)}</exif:GPSLongitude>
+        <exif:GPSLatitude>${abs(latitude)}</exif:GPSLatitude>
+        <exif:GPSLatitudeRef>${"N" if latitude >= 0 else "S"}</exif:GPSLatitudeRef>
+    % endif
+</%def>
+
 <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
     <!-- mirrors Photos 5 "Export IPTC as XMP" option -->
     <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
         <rdf:Description rdf:about="" 
             xmlns:dc="http://purl.org/dc/elements/1.1/" 
             xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
-            ${dc_description(photo.description)}
+            ${photoshop_sidecar_for_extension(extension)}
+            ${dc_description(description)}
             ${dc_title(photo.title)}
             ${dc_subject(subjects)}
             ${dc_datecreated(photo.date)}
         </rdf:Description>
-        <rdf:Description rdf:about='' 
+        <rdf:Description rdf:about=""  
             xmlns:Iptc4xmpExt='http://iptc.org/std/Iptc4xmpExt/2008-02-29/'>
             ${iptc_personinimage(persons)}
         </rdf:Description>
-        <rdf:Description rdf:about='' 
+        <rdf:Description rdf:about="" 
             xmlns:digiKam='http://www.digikam.org/ns/1.0/'>
             ${dk_tagslist(keywords)}
         </rdf:Description>
-        <rdf:Description rdf:about='' 
+        <rdf:Description rdf:about="" 
             xmlns:xmp='http://ns.adobe.com/xap/1.0/'>
             ${adobe_createdate(photo.date)}
             ${adobe_modifydate(photo.date)}
         </rdf:Description>
+        <rdf:Description rdf:about=""
+            xmlns:exif='http://ns.adobe.com/exif/1.0/'>
+            ${gps_info(*photo.location)}
+        </rdf:Description>
    </rdf:RDF>
 </x:xmpmeta>

osxphotos/utils.py

@@ -10,6 +10,7 @@ import sqlite3
 import subprocess
 import sys
 import tempfile
+import unicodedata
 import urllib.parse
 from plistlib import load as plistload
 
@@ -18,6 +19,7 @@ import CoreServices
 import objc
 from Foundation import *
 
+from ._constants import UNICODE_FORMAT
 from .fileutil import FileUtil
 
 _DEBUG = False
@@ -262,7 +264,10 @@ def get_preferred_uti_extension(uti):
 
 def findfiles(pattern, path_):
     """Returns list of filenames from path_ matched by pattern
-       shell pattern. Matching is case-insensitive."""
+       shell pattern. Matching is case-insensitive.
+       If 'path_' is invalid/doesn't exist, returns []."""
+    if not os.path.isdir(path_):
+        return []
     # See: https://gist.github.com/techtonik/5694830
 
     rule = re.compile(fnmatch.translate(pattern), re.IGNORECASE)
@@ -349,3 +354,13 @@ def _db_is_locked(dbname):
 #     attr = xattr.xattr(filepath)
 #     uuid_bytes = bytes(uuid, 'utf-8')
 #     attr.set(OSXPHOTOS_XATTR_UUID, uuid_bytes)
+
+
+def normalize_unicode(value):
+    """ normalize unicode data """
+    if value is not None:
+        if not isinstance(value, str):
+            raise ValueError("value must be str")
+        return unicodedata.normalize(UNICODE_FORMAT, value)
+    else:
+        return None

requirements.txt

@@ -1,19 +1,38 @@
+aiohttp==4.0.0a1
 altgraph==0.17
 ansimarkup==1.4.0
 appdirs==1.4.3
+appnope==0.1.0
 astroid==2.2.5
+async-timeout==3.0.1
 atomicwrites==1.3.0
 attrs==19.1.0
+backcall==0.1.0
 better-exceptions-fork==0.2.1.post6
-# bpylist2==2.0.3;python_version<"3.8"
-https://github.com/RhetTbull/bpylist/releases/download/v2.0.3/bpylist2-2.0.3.tar.gz#egg=bpylist2;python_version<"3.8"
-bpylist2==3.0.0;python_version>="3.8"
-certifi==2019.3.9
+black==19.10b0
+bleach==3.1.4
+bpylist2==3.0.2
+certifi==2020.4.5.1
+cffi==1.14.0
+chardet==3.0.4
 Click==7.0
 colorama==0.4.1
 coverage==4.5.4
-importlib-metadata>=0.18
+decorator==4.4.2
+distlib==0.3.1
+docutils==0.16
+entrypoints==0.3
+filelock==3.0.12
+idna==2.9
+importlib-metadata==1.6.0
+ipykernel==5.1.4
+ipython==7.13.0
+ipython-genutils==0.2.0
 isort==4.3.20
+jedi==0.16.0
+jupyter-client==6.1.2
+jupyter-core==4.6.3
+keyring==21.2.0
 lazy-object-proxy==1.4.1
 loguru==0.2.5
 macholib==1.14
@@ -22,135 +41,166 @@ MarkupSafe==1.1.1
 mccabe==0.6.1
 modulegraph==0.18
 more-itertools==7.2.0
+multidict==4.7.6
 packaging==19.0
+parso==0.6.2
 pathspec==0.7.0
 pathvalidate==2.2.1
+pexpect==4.8.0
+pickleshare==0.7.5
+Pillow==7.2.0
+pkginfo==1.5.0.1
 pluggy==0.12.0
+prompt-toolkit==3.0.4
+psutil==5.7.0
+ptyprocess==0.6.0
 py==1.8.0
 py2app==0.21
-Pygments==2.4.2
+pycparser==2.20
+pyfiglet==0.8.post1
+Pygments==2.6.1
+PyInstaller==3.6
+pyinstaller-setuptools==2019.3
 pylint==2.3.1
-pyobjc==6.0.1
-pyobjc-core==6.0.1
-pyobjc-framework-Accounts==6.0.1
-pyobjc-framework-AddressBook==6.0.1
-pyobjc-framework-AdSupport==6.0.1
-pyobjc-framework-AppleScriptKit==6.0.1
-pyobjc-framework-AppleScriptObjC==6.0.1
-pyobjc-framework-ApplicationServices==6.0.1
-pyobjc-framework-AuthenticationServices==6.0.1
-pyobjc-framework-Automator==6.0.1
-pyobjc-framework-AVFoundation==6.0.1
-pyobjc-framework-AVKit==6.0.1
-pyobjc-framework-BusinessChat==6.0.1
-pyobjc-framework-CalendarStore==6.0.1
-pyobjc-framework-CFNetwork==6.0.1
-pyobjc-framework-CloudKit==6.0.1
-pyobjc-framework-Cocoa==6.0.1
-pyobjc-framework-Collaboration==6.0.1
-pyobjc-framework-ColorSync==6.0.1
-pyobjc-framework-Contacts==6.0.1
-pyobjc-framework-ContactsUI==6.0.1
-pyobjc-framework-CoreAudio==6.0.1
-pyobjc-framework-CoreAudioKit==6.0.1
-pyobjc-framework-CoreBluetooth==6.0.1
-pyobjc-framework-CoreData==6.0.1
-pyobjc-framework-CoreHaptics==6.0.1
-pyobjc-framework-CoreLocation==6.0.1
-pyobjc-framework-CoreMedia==6.0.1
-pyobjc-framework-CoreMediaIO==6.0.1
-pyobjc-framework-CoreML==6.0.1
-pyobjc-framework-CoreMotion==6.0.1
-pyobjc-framework-CoreServices==6.0.1
-pyobjc-framework-CoreSpotlight==6.0.1
-pyobjc-framework-CoreText==6.0.1
-pyobjc-framework-CoreWLAN==6.0.1
-pyobjc-framework-CryptoTokenKit==6.0.1
-pyobjc-framework-DeviceCheck==6.0.1
-pyobjc-framework-DictionaryServices==6.0.1
-pyobjc-framework-DiscRecording==6.0.1
-pyobjc-framework-DiscRecordingUI==6.0.1
-pyobjc-framework-DiskArbitration==6.0.1
-pyobjc-framework-DVDPlayback==6.0.1
-pyobjc-framework-EventKit==6.0.1
-pyobjc-framework-ExceptionHandling==6.0.1
-pyobjc-framework-ExecutionPolicy==6.0.1
-pyobjc-framework-ExternalAccessory==6.0.1
-pyobjc-framework-FileProvider==6.0.1
-pyobjc-framework-FileProviderUI==6.0.1
-pyobjc-framework-FinderSync==6.0.1
-pyobjc-framework-FSEvents==6.0.1
-pyobjc-framework-GameCenter==6.0.1
-pyobjc-framework-GameController==6.0.1
-pyobjc-framework-GameKit==6.0.1
-pyobjc-framework-GameplayKit==6.0.1
-pyobjc-framework-ImageCaptureCore==6.0.1
-pyobjc-framework-IMServicePlugIn==6.0.1
-pyobjc-framework-InputMethodKit==6.0.1
-pyobjc-framework-InstallerPlugins==6.0.1
-pyobjc-framework-InstantMessage==6.0.1
-pyobjc-framework-Intents==6.0.1
-pyobjc-framework-IOSurface==6.0.1
-pyobjc-framework-iTunesLibrary==6.0.1
-pyobjc-framework-LatentSemanticMapping==6.0.1
-pyobjc-framework-LaunchServices==6.0.1
-pyobjc-framework-libdispatch==6.0.1
-pyobjc-framework-LinkPresentation==6.0.1
-pyobjc-framework-LocalAuthentication==6.0.1
-pyobjc-framework-MapKit==6.0.1
-pyobjc-framework-MediaAccessibility==6.0.1
-pyobjc-framework-MediaLibrary==6.0.1
-pyobjc-framework-MediaPlayer==6.0.1
-pyobjc-framework-MediaToolbox==6.0.1
-pyobjc-framework-MetalKit==6.0.1
-pyobjc-framework-ModelIO==6.0.1
-pyobjc-framework-MultipeerConnectivity==6.0.1
-pyobjc-framework-NaturalLanguage==6.0.1
-pyobjc-framework-NetFS==6.0.1
-pyobjc-framework-Network==6.0.1
-pyobjc-framework-NetworkExtension==6.0.1
-pyobjc-framework-NotificationCenter==6.0.1
-pyobjc-framework-OpenDirectory==6.0.1
-pyobjc-framework-OSAKit==6.0.1
-pyobjc-framework-OSLog==6.0.1
-pyobjc-framework-PencilKit==6.0.1
-pyobjc-framework-Photos==6.0.1
-pyobjc-framework-PhotosUI==6.0.1
-pyobjc-framework-PreferencePanes==6.0.1
-pyobjc-framework-PubSub==6.0.1
-pyobjc-framework-PushKit==6.0.1
+pyobjc==6.2.2
+pyobjc-core==6.2.2
+pyobjc-framework-Accounts==6.2.2
+pyobjc-framework-AddressBook==6.2.2
+pyobjc-framework-AdSupport==6.2.2
+pyobjc-framework-AppleScriptKit==6.2.2
+pyobjc-framework-AppleScriptObjC==6.2.2
+pyobjc-framework-ApplicationServices==6.2.2
+pyobjc-framework-AuthenticationServices==6.2.2
+pyobjc-framework-AutomaticAssessmentConfiguration==6.2.2
+pyobjc-framework-Automator==6.2.2
+pyobjc-framework-AVFoundation==6.2.2
+pyobjc-framework-AVKit==6.2.2
+pyobjc-framework-BusinessChat==6.2.2
+pyobjc-framework-CalendarStore==6.2.2
+pyobjc-framework-CFNetwork==6.2.2
+pyobjc-framework-CloudKit==6.2.2
+pyobjc-framework-Cocoa==6.2.2
+pyobjc-framework-Collaboration==6.2.2
+pyobjc-framework-ColorSync==6.2.2
+pyobjc-framework-Contacts==6.2.2
+pyobjc-framework-ContactsUI==6.2.2
+pyobjc-framework-CoreAudio==6.2.2
+pyobjc-framework-CoreAudioKit==6.2.2
+pyobjc-framework-CoreBluetooth==6.2.2
+pyobjc-framework-CoreData==6.2.2
+pyobjc-framework-CoreHaptics==6.2.2
+pyobjc-framework-CoreLocation==6.2.2
+pyobjc-framework-CoreMedia==6.2.2
+pyobjc-framework-CoreMediaIO==6.2.2
+pyobjc-framework-CoreML==6.2.2
+pyobjc-framework-CoreMotion==6.2.2
+pyobjc-framework-CoreServices==6.2.2
+pyobjc-framework-CoreSpotlight==6.2.2
+pyobjc-framework-CoreText==6.2.2
+pyobjc-framework-CoreWLAN==6.2.2
+pyobjc-framework-CryptoTokenKit==6.2.2
+pyobjc-framework-DeviceCheck==6.2.2
+pyobjc-framework-DictionaryServices==6.2.2
+pyobjc-framework-DiscRecording==6.2.2
+pyobjc-framework-DiscRecordingUI==6.2.2
+pyobjc-framework-DiskArbitration==6.2.2
+pyobjc-framework-DVDPlayback==6.2.2
+pyobjc-framework-EventKit==6.2.2
+pyobjc-framework-ExceptionHandling==6.2.2
+pyobjc-framework-ExecutionPolicy==6.2.2
+pyobjc-framework-ExternalAccessory==6.2.2
+pyobjc-framework-FileProvider==6.2.2
+pyobjc-framework-FileProviderUI==6.2.2
+pyobjc-framework-FinderSync==6.2.2
+pyobjc-framework-FSEvents==6.2.2
+pyobjc-framework-GameCenter==6.2.2
+pyobjc-framework-GameController==6.2.2
+pyobjc-framework-GameKit==6.2.2
+pyobjc-framework-GameplayKit==6.2.2
+pyobjc-framework-ImageCaptureCore==6.2.2
+pyobjc-framework-IMServicePlugIn==6.2.2
+pyobjc-framework-InputMethodKit==6.2.2
+pyobjc-framework-InstallerPlugins==6.2.2
+pyobjc-framework-InstantMessage==6.2.2
+pyobjc-framework-Intents==6.2.2
+pyobjc-framework-IOSurface==6.2.2
+pyobjc-framework-iTunesLibrary==6.2.2
+pyobjc-framework-LatentSemanticMapping==6.2.2
+pyobjc-framework-LaunchServices==6.2.2
+pyobjc-framework-libdispatch==6.2.2
+pyobjc-framework-LinkPresentation==6.2.2
+pyobjc-framework-LocalAuthentication==6.2.2
+pyobjc-framework-MapKit==6.2.2
+pyobjc-framework-MediaAccessibility==6.2.2
+pyobjc-framework-MediaLibrary==6.2.2
+pyobjc-framework-MediaPlayer==6.2.2
+pyobjc-framework-MediaToolbox==6.2.2
+pyobjc-framework-Metal==6.2.2
+pyobjc-framework-MetalKit==6.2.2
+pyobjc-framework-ModelIO==6.2.2
+pyobjc-framework-MultipeerConnectivity==6.2.2
+pyobjc-framework-NaturalLanguage==6.2.2
+pyobjc-framework-NetFS==6.2.2
+pyobjc-framework-Network==6.2.2
+pyobjc-framework-NetworkExtension==6.2.2
+pyobjc-framework-NotificationCenter==6.2.2
+pyobjc-framework-OpenDirectory==6.2.2
+pyobjc-framework-OSAKit==6.2.2
+pyobjc-framework-OSLog==6.2.2
+pyobjc-framework-PencilKit==6.2.2
+pyobjc-framework-Photos==6.2.2
+pyobjc-framework-PhotosUI==6.2.2
+pyobjc-framework-PreferencePanes==6.2.2
+pyobjc-framework-PubSub==6.2
+pyobjc-framework-PushKit==6.2.2
 pyobjc-framework-QTKit==6.0.1
-pyobjc-framework-Quartz==6.0.1
-pyobjc-framework-QuickLookThumbnailing==6.0.1
-pyobjc-framework-SafariServices==6.0.1
-pyobjc-framework-SceneKit==6.0.1
-pyobjc-framework-ScreenSaver==6.0.1
-pyobjc-framework-ScriptingBridge==6.0.1
-pyobjc-framework-SearchKit==6.0.1
-pyobjc-framework-Security==6.0.1
-pyobjc-framework-SecurityFoundation==6.0.1
-pyobjc-framework-SecurityInterface==6.0.1
-pyobjc-framework-ServiceManagement==6.0.1
-pyobjc-framework-Social==6.0.1
-pyobjc-framework-SoundAnalysis==6.0.1
-pyobjc-framework-Speech==6.0.1
-pyobjc-framework-SpriteKit==6.0.1
-pyobjc-framework-StoreKit==6.0.1
-pyobjc-framework-SyncServices==6.0.1
-pyobjc-framework-SystemConfiguration==6.0.1
-pyobjc-framework-SystemExtensions==6.0.1
-pyobjc-framework-UserNotifications==6.0.1
-pyobjc-framework-VideoSubscriberAccount==6.0.1
-pyobjc-framework-VideoToolbox==6.0.1
-pyobjc-framework-Vision==6.0.1
-pyobjc-framework-WebKit==6.0.1
+pyobjc-framework-Quartz==6.2.2
+pyobjc-framework-QuickLookThumbnailing==6.2.2
+pyobjc-framework-SafariServices==6.2.2
+pyobjc-framework-SceneKit==6.2.2
+pyobjc-framework-ScreenSaver==6.2.2
+pyobjc-framework-ScriptingBridge==6.2.2
+pyobjc-framework-SearchKit==6.2.2
+pyobjc-framework-Security==6.2.2
+pyobjc-framework-SecurityFoundation==6.2.2
+pyobjc-framework-SecurityInterface==6.2.2
+pyobjc-framework-ServiceManagement==6.2.2
+pyobjc-framework-Social==6.2.2
+pyobjc-framework-SoundAnalysis==6.2.2
+pyobjc-framework-Speech==6.2.2
+pyobjc-framework-SpriteKit==6.2.2
+pyobjc-framework-StoreKit==6.2.2
+pyobjc-framework-SyncServices==6.2.2
+pyobjc-framework-SystemConfiguration==6.2.2
+pyobjc-framework-SystemExtensions==6.2.2
+pyobjc-framework-UserNotifications==6.2.2
+pyobjc-framework-VideoSubscriberAccount==6.2.2
+pyobjc-framework-VideoToolbox==6.2.2
+pyobjc-framework-Vision==6.2.2
+pyobjc-framework-WebKit==6.2.2
 pyparsing==2.4.1.1
+python-dateutil==2.8.1
 PyYAML==5.1.2
+pyzmq==18.1.1
+readme-renderer==25.0
 regex==2020.2.20
-six==1.12.0
+requests==2.23.0
+requests-toolbelt==0.9.1
+six==1.14.0
 termcolor==1.1.0
 toml==0.10.0
+tornado==6.0.4
+tox==3.19.0
+tox-conda==0.2.1
+tqdm==4.45.0
+traitlets==4.3.3
+twine==3.1.1
 typed-ast==1.4.1
-wcwidth==0.1.7
+typing-extensions==3.7.4.2
+urllib3==1.25.9
+virtualenv==20.0.30
+wcwidth==0.1.9
+webencodings==0.5.1
 wrapt==1.11.1
+yarl==1.4.2
 zipp==0.5.2

setup.py

@@ -48,17 +48,6 @@ with open(
 with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
     about["long_description"] = f.read()
 
-# ugly hack to install custom version of bpylist2 needed for Python < 3.8
-# the stock version of bylist2==2.0.3 causes an error related to
-# "pkg_resources.ContextualVersionConflict: (pycodestyle 2.3.1..."
-# PEP 508 no help here as URL-based lookups not allowed in PyPI packages
-# if you know a better way, PRs welcome!
-# once I go to 3.8+ required, this won't be necessary as bpylist2 3.0+ solves this issue
-if py_ver < 3.8:
-    os.system(
-        "python3 -m pip install git+git://github.com/RhetTbull/bpylist2.git#egg=bpylist2"
-    )
-
 setup(
     name="osxphotos",
     version=about["__version__"],
@@ -78,16 +67,15 @@ setup(
         "Intended Audience :: Developers",
         "License :: OSI Approved :: MIT License",
         "Operating System :: MacOS :: MacOS X",
-        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.7",
         "Topic :: Software Development :: Libraries :: Python Modules",
     ],
     install_requires=[
-        "pyobjc>=6.0.1",
+        "pyobjc>=6.2.2",
         "Click>=7",
         "PyYAML>=5.1.2",
         "Mako>=1.1.1",
-        "bpylist2==2.0.3;python_version<'3.8'",
-        "bpylist2==3.0.0;python_version>='3.8'",
+        "bpylist2==3.0.2",
         "pathvalidate==2.2.1",
         "dataclasses==0.7;python_version<'3.7'",
     ],

tests/README.md

@@ -17,14 +17,29 @@ Some of the export tests rely on photos in my local library and will look for `O
 
 One test for locale does not run on GitHub's automated workflow and will look for `OSXPHOTOS_TEST_LOCALE=1` to determine if it should be run.  If you want to run this test, set the environment variable.  
 
-## Attribution ##
-These tests utilize a test Photos library. The test library is populated with photos from [flickr](https://www.flickr.com).  All images used are licensed under Creative Commons 2.0 Attribution [license](https://creativecommons.org/licenses/by/2.0/).  
+## Test Photo Libraries
+**Important**: The test code uses several test photo libraries created on various version of MacOS.  If you need to inspect one of these or modify one for a test, make a copy of the library (for example, copy it to your ~/Pictures folder) then open the copy in Photos.  Once done, copy the revised library back to the tests/ folder.  If you do not do this, the Photos background process photoanalysisd will forever try to process the library resulting in updates to the database which will cause git to see changes to the file you didn't intend.  I'm not aware of any way to disassociate photoanalysisd from the library once you've opened it in Photos.
 
-Images used from:
+## Attribution ##
+These tests utilize a test Photos library. The test library is populated with photos from [flickr](https://www.flickr.com) and from my own photo library.  All images used are licensed under Creative Commons 2.0 Attribution [license](https://creativecommons.org/licenses/by/2.0/).  
+
+Flickr images used from:
 - [Jeff Hitchcock](https://www.flickr.com/photos/arbron/48353451872/)
 - [Carlos Montesdeoca](https://www.flickr.com/photos/carlosmontesdeocastudio)
 - [Rydale Clothing](https://www.flickr.com/photos/rydaleclothing)
 - [Marco Verch](https://www.flickr.com/photos/30478819@N08/48228222317/)
 - [K M](https://www.flickr.com/photos/153387643@N08/49334338022/)
-
+- [Shelby Mash](https://www.flickr.com/photos/shelbzyleigh/3809603052)
+- [Rory MacLeod](https://www.flickr.com/photos/macrj/6969547134)
+- [Md. Al Amin](https://www.flickr.com/photos/alamin_bd/45207044465)
+- [Fatlum Haliti](https://www.flickr.com/photos/lumlumi/363449752)
+- [Benny Mazur](https://www.flickr.com/photos/benimoto/399012465)
+- [Sara Cooper PR](https://www.flickr.com/photos/saracooperpr/6422472677)
+- [herval](https://www.flickr.com/photos/herval/2403994289)
+- [Vox Efx](https://www.flickr.com/photos/vox_efx/141137669)
+- [Bill Strain](https://www.flickr.com/photos/billstrain/5117042252)
+- [Guilherme Yagui](https://www.flickr.com/photos/yagui7/15895161088/)
+- [Deborah Austin](https://www.flickr.com/photos/littledebbie11/8703591799/)
+- [We Are Social](https://www.flickr.com/photos/wearesocial/23309711462/)
+- [cloud.shepherd](https://www.flickr.com/photos/exnucboy1/31017877125)
 

tests/Test-10.14.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-04-25T23:54:43Z</date>
+	<date>2020-07-27T03:16:28Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-06-27T16:03:48Z</date>
+	<date>2020-07-27T12:35:43Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/private/com.apple.photomodel/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2020-04-17T17:51:16Z</date>
+		<date>2020-07-27T03:18:40Z</date>
 	</dict>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2020-06-27T16:03:43Z</date>
+	<date>2020-07-27T03:16:25Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>651</integer>
+	<integer>707</integer>
 	<key>LibraryBuildTag</key>
 	<string>D8C4AAA1-3AB6-4A65-BEBD-99CC3E5D433E</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.14.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>606</integer>
+		<integer>707</integer>
 		<key>LibraryBuildTag</key>
 		<string>D8C4AAA1-3AB6-4A65-BEBD-99CC3E5D433E</string>
 		<key>LibrarySchemaVersion</key>
@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-27T13:16:43Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2020-06-27T16:03:33Z</date>
+	<date>2020-07-27T03:18:40Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>

tests/Test-10.15.5.photoslibrary/database/Photos.sqlite.lock

@@ -7,7 +7,7 @@
 	<key>hostuuid</key>
 	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
 	<key>pid</key>
-	<integer>763</integer>
+	<integer>3125</integer>
 	<key>processname</key>
 	<string>photolibraryd</string>
 	<key>uid</key>

tests/Test-10.15.6.photoslibrary/database/DataModelVersion.plist

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>LibrarySchemaVersion</key>
+	<integer>5001</integer>
+	<key>MetaSchemaVersion</key>
+	<integer>3</integer>
+</dict>
+</plist>