Fixed handling of date_modified for Catalina, issue #247

Convert to jpeg

.github/workflows/pythonpackage.yml

@@ -1,6 +1,6 @@
-name: Python package
+name: Tests
 
-on: [push]
+on: [push, pull_request]
 
 jobs:
   build:
@@ -9,7 +9,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [3.8]
+        python-version: [3.7, 3.8]
 
     steps:
     - uses: actions/checkout@v1

CHANGELOG.md

@@ -4,6 +4,359 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.36.0](https://github.com/RhetTbull/osxphotos/compare/v0.35.7...v0.36.0)
+
+> 26 October 2020
+
+- Added verbose to PhotosDB(), partial fix for #110 [`d87b8f3`](https://github.com/RhetTbull/osxphotos/commit/d87b8f30a45cbb6fdb315a12f8585e2bdc21be6b)
+- Added comments/likes, implements #214 [`23de6b5`](https://github.com/RhetTbull/osxphotos/commit/23de6b58908371d9ca55d1d1999c6d56de454180)
+- Cleaned up constructor for PhotosDB [`667c89e`](https://github.com/RhetTbull/osxphotos/commit/667c89e32c3f96baeafebc03e83517ea05693b00)
+
+#### [v0.35.7](https://github.com/RhetTbull/osxphotos/compare/v0.35.6...v0.35.7)
+
+> 24 October 2020
+
+- Fix for issue #238 [`48f29e1`](https://github.com/RhetTbull/osxphotos/commit/48f29e138e4e9da3eba78f3681ee9b8cb28910df)
+
+#### [v0.35.6](https://github.com/RhetTbull/osxphotos/compare/v0.35.5...v0.35.6)
+
+> 24 October 2020
+
+- Fixed shared, not_shared in cli [`8551981`](https://github.com/RhetTbull/osxphotos/commit/8551981f68f0cd2a3a081cc21ae287ff981b9b4b)
+
+#### [v0.35.5](https://github.com/RhetTbull/osxphotos/compare/v0.35.4...v0.35.5)
+
+> 22 October 2020
+
+- Added get_shared_photo_comments.py to examples [`15e0914`](https://github.com/RhetTbull/osxphotos/commit/15e0914af6301a945bc751173aef6718487d9637)
+- Fix for issue #237 [`a416de2`](https://github.com/RhetTbull/osxphotos/commit/a416de29e4ac39a5c323f7913b05a8c38ad205be)
+- Added test for issue #235 [`ea68229`](https://github.com/RhetTbull/osxphotos/commit/ea68229ddac2e2301ac2d5607451cf7d00207d5d)
+
+#### [v0.35.4](https://github.com/RhetTbull/osxphotos/compare/v0.35.3...v0.35.4)
+
+> 18 October 2020
+
+- refactored template code to fix #213 [`#213`](https://github.com/RhetTbull/osxphotos/issues/213)
+
+#### [v0.35.3](https://github.com/RhetTbull/osxphotos/compare/v0.35.2...v0.35.3)
+
+> 15 October 2020
+
+- Fix for issue #235, #236 [`41b2399`](https://github.com/RhetTbull/osxphotos/commit/41b23991df3d1d553b70889ede237f83b6874519)
+
+#### [v0.35.2](https://github.com/RhetTbull/osxphotos/compare/v0.35.1...v0.35.2)
+
+> 12 October 2020
+
+- Fix for issue #234 [`da100f9`](https://github.com/RhetTbull/osxphotos/commit/da100f93a9b849ca4750336d7f90e9023e39dd07)
+
+#### [v0.35.1](https://github.com/RhetTbull/osxphotos/compare/v0.35.0...v0.35.1)
+
+> 12 October 2020
+
+- Fix for issue #230 [`dcbf8f2`](https://github.com/RhetTbull/osxphotos/commit/dcbf8f25f61e21bcf1040046aa9d6ddba4ac9735)
+
+#### [v0.35.0](https://github.com/RhetTbull/osxphotos/compare/v0.34.5...v0.35.0)
+
+> 12 October 2020
+
+- Convert to jpeg [`#233`](https://github.com/RhetTbull/osxphotos/pull/233)
+- Updated tests, closes #231 [`#231`](https://github.com/RhetTbull/osxphotos/issues/231)
+- Updated tests [`b0171ba`](https://github.com/RhetTbull/osxphotos/commit/b0171ba6f5b73e1ff71e16d27852f8df7f208f60)
+- Updated tests [`07b0843`](https://github.com/RhetTbull/osxphotos/commit/07b08433df5a60f191e23a95394e83e51dca016f)
+- Merge branch 'master' into convert_to_jpeg [`fe5185b`](https://github.com/RhetTbull/osxphotos/commit/fe5185be8893002da663039f8ec103faed0f1831)
+- Added israw, tests for Big Sur [`b5a9794`](https://github.com/RhetTbull/osxphotos/commit/b5a9794f6bff5683fd42a22197454940e4d7ba88)
+- Updates to path, path_raw, uti for RAW+JPEG pairs [`b32f4b8`](https://github.com/RhetTbull/osxphotos/commit/b32f4b8504768a5f4b5ad54c00315b9e82fca980)
+
+#### [v0.34.5](https://github.com/RhetTbull/osxphotos/compare/v0.34.3...v0.34.5)
+
+> 6 October 2020
+
+- --convert-to-jpeg initial version working [`38f201d`](https://github.com/RhetTbull/osxphotos/commit/38f201d0fb70bf299a828c1dd0d034a119e380c4)
+- Added tests, fixed bug in export_db [`5a13605`](https://github.com/RhetTbull/osxphotos/commit/5a13605f850bb947c8888246f06a5ca4e6aa5f10)
+- Updated tests [`b2b39aa`](https://github.com/RhetTbull/osxphotos/commit/b2b39aa6075df11861cf5d8945b657204f120e87)
+
+#### [v0.34.3](https://github.com/RhetTbull/osxphotos/compare/v0.34.2...v0.34.3)
+
+> 29 September 2020
+
+- Update exiftool.py to preserve file modification time, thanks to @hhoeck  [`#223`](https://github.com/RhetTbull/osxphotos/pull/223)
+- Added tests for 10.15.6 [`432da7f`](https://github.com/RhetTbull/osxphotos/commit/432da7f139a5e4b37eeb358f4ede45314407f8e5)
+- Added HEIC test image [`ddc1e69`](https://github.com/RhetTbull/osxphotos/commit/ddc1e69b4a4ac712e1af312b865c4216f9ad350c)
+- Fixed bug related to issue #222 [`c939df7`](https://github.com/RhetTbull/osxphotos/commit/c939df717159e8b97955c0b267327cd56a9ed56c)
+- Version bump for bug fix [`62d54cc`](https://github.com/RhetTbull/osxphotos/commit/62d54cc0beabd0141545608184d4b2c658eedf0f)
+- Update README.md [`6883fec`](https://github.com/RhetTbull/osxphotos/commit/6883fec2b2236d892b88327e1b4e9da1237f7dea)
+
+#### [v0.34.2](https://github.com/RhetTbull/osxphotos/compare/v0.34.1...v0.34.2)
+
+> 14 September 2020
+
+- Partial fix for issue #213 [`459d91d`](https://github.com/RhetTbull/osxphotos/commit/459d91d7b11dbd4b0564906c1689b60dc5b64642)
+
+#### [v0.34.1](https://github.com/RhetTbull/osxphotos/compare/v0.34.0...v0.34.1)
+
+> 13 September 2020
+
+- Fixed exception handling in export [`eb00ffd`](https://github.com/RhetTbull/osxphotos/commit/eb00ffd73737ef4832229e4e6fd8dc4ccb0b8539)
+- Updated README.md [`a1776fa`](https://github.com/RhetTbull/osxphotos/commit/a1776fa14850275ad6b02ece80bbe8ce908fa836)
+
+#### [v0.34.0](https://github.com/RhetTbull/osxphotos/compare/v0.33.8...v0.34.0)
+
+> 7 September 2020
+
+- Added --skip-original-if-edited for issue #159 [`5f2d401`](https://github.com/RhetTbull/osxphotos/commit/5f2d401048850fd68f31b37a7e71abc11ca80dc5)
+- Still working on issue #208 [`58b3869`](https://github.com/RhetTbull/osxphotos/commit/58b3869a7cce7cb3f211599e544d7e5426ceb4a6)
+
+#### [v0.33.8](https://github.com/RhetTbull/osxphotos/compare/v0.33.7...v0.33.8)
+
+> 31 August 2020
+
+- Fixed sidecar collisions, closes #210 [`#210`](https://github.com/RhetTbull/osxphotos/issues/210)
+
+#### [v0.33.7](https://github.com/RhetTbull/osxphotos/compare/v0.33.5...v0.33.7)
+
+> 31 August 2020
+
+- typo fix - thanks to @dmd  [`#212`](https://github.com/RhetTbull/osxphotos/pull/212)
+- Normalize unicode for issue #208 [`a36eb41`](https://github.com/RhetTbull/osxphotos/commit/a36eb416b19284477922b6a5f837f4040327138b)
+- Added force_download.py to examples [`b611d34`](https://github.com/RhetTbull/osxphotos/commit/b611d34d19db480af72f57ef55eacd0a32c8d1e8)
+- Added photoshop:SidecarForExtension to XMP, partial fix for #210 [`60d96a8`](https://github.com/RhetTbull/osxphotos/commit/60d96a8f563882fba2365a6ab58c1276725eedaa)
+- Updated README.md [`c9b1518`](https://github.com/RhetTbull/osxphotos/commit/c9b15186a022d91248451279e5f973e3f2dca4b4)
+- Update README.md [`42e8fba`](https://github.com/RhetTbull/osxphotos/commit/42e8fba125a3c6b1bd0d538f2af511aabfbeb478)
+
+#### [v0.33.5](https://github.com/RhetTbull/osxphotos/compare/v0.33.3...v0.33.5)
+
+> 25 August 2020
+
+- Fixed DST handling for from_date/to_date, closes #193 (again) [`#193`](https://github.com/RhetTbull/osxphotos/issues/193)
+- Added raw timestamps to PhotoInfo._info [`0f457a4`](https://github.com/RhetTbull/osxphotos/commit/0f457a4082a4eebc42a5df2160a02ad987b6f96c)
+
+#### [v0.33.3](https://github.com/RhetTbull/osxphotos/compare/v0.33.2...v0.33.3)
+
+> 23 August 2020
+
+- Fixed portrait for Catalina/Big Sur; see issue #203 [`1f717b0`](https://github.com/RhetTbull/osxphotos/commit/1f717b05794c2088c7c15d2aab0c5d24b6309c06)
+
+#### [v0.33.2](https://github.com/RhetTbull/osxphotos/compare/v0.33.0...v0.33.2)
+
+> 23 August 2020
+
+- Closes issue #206, adds --touch-file [`#207`](https://github.com/RhetTbull/osxphotos/pull/207)
+- Touch files - fixes #194 -- thanks to @PabloKohan  [`#205`](https://github.com/RhetTbull/osxphotos/pull/205)
+- Refactor/cleanup _export_photo - thanks to @PabloKohan  [`#204`](https://github.com/RhetTbull/osxphotos/pull/204)
+- Finished --touch-file, closes #206 [`#206`](https://github.com/RhetTbull/osxphotos/issues/206)
+- Merge pull request #205 from PabloKohan/touch_files__fix_194 [`#194`](https://github.com/RhetTbull/osxphotos/issues/194)
+- --touch-file now working with --update [`6c11e3f`](https://github.com/RhetTbull/osxphotos/commit/6c11e3fa5b5b05b98b9fdbb0e59e3a78c7dff980)
+- Refactor/cleanup _export_photo [`eefa1f1`](https://github.com/RhetTbull/osxphotos/commit/eefa1f181f4fd7b027ae69abd2b764afb590c081)
+- Fixed touch tests [`1bf7105`](https://github.com/RhetTbull/osxphotos/commit/1bf7105737fbd756064a2f9ef4d4bbd0b067978c)
+- Working on issue 206 [`ebd878a`](https://github.com/RhetTbull/osxphotos/commit/ebd878a075983ef3df0b1ead1a725e01508721f8)
+- Working on issue #206 [`c9c9202`](https://github.com/RhetTbull/osxphotos/commit/c9c920220545dc27c8cb1379d7bde15987cce72c)
+
+#### [v0.33.0](https://github.com/RhetTbull/osxphotos/compare/v0.32.0...v0.33.0)
+
+> 17 August 2020
+
+- Replaced call to which, closes #171 [`#171`](https://github.com/RhetTbull/osxphotos/issues/171)
+- Added contributors to README.md, closes #200 [`#200`](https://github.com/RhetTbull/osxphotos/issues/200)
+- Added tests for 10.15.6 [`d2deeff`](https://github.com/RhetTbull/osxphotos/commit/d2deefff834e46e1a26adc01b1b025ac839dbc78)
+- Added ImportInfo for Photos 5+ [`98e4170`](https://github.com/RhetTbull/osxphotos/commit/98e417023ec5bd8292b25040d0844f3706645950)
+- Update README.md [`360c8d8`](https://github.com/RhetTbull/osxphotos/commit/360c8d8e1b4760e95a8b71b3a0bf0df4fb5adaf5)
+- Update README.md [`868cda8`](https://github.com/RhetTbull/osxphotos/commit/868cda8482ce6b29dd00e04a209d40550e6b128b)
+
+#### [v0.32.0](https://github.com/RhetTbull/osxphotos/compare/v0.31.2...v0.32.0)
+
+> 9 August 2020
+
+- Alpha support for MacOS Big Sur/10.16, see issue #187 [`6acf9ac`](https://github.com/RhetTbull/osxphotos/commit/6acf9acd6364e1996158179493d128ec0958e652)
+
+#### [v0.31.2](https://github.com/RhetTbull/osxphotos/compare/v0.31.0...v0.31.2)
+
+> 9 August 2020
+
+- Fixed from_date and to_date to be timezone aware, closes #193 [`#193`](https://github.com/RhetTbull/osxphotos/issues/193)
+- Added test for valid XMP file, closes #197 [`#197`](https://github.com/RhetTbull/osxphotos/issues/197)
+- Dropped py36 due to datetime.fromisoformat [`a714ae0`](https://github.com/RhetTbull/osxphotos/commit/a714ae0af089b13acf70c4f29934393aa48ed222)
+- Added --uuid-from-file to CLI [`840e993`](https://github.com/RhetTbull/osxphotos/commit/840e9937bede407ef55972a361618683245e086b)
+- Added write_uuid_to_file.applescript to utils [`bea770b`](https://github.com/RhetTbull/osxphotos/commit/bea770b322d21cf3f8245d20e182006247cb71d6)
+- Updated README.md [`002fce8`](https://github.com/RhetTbull/osxphotos/commit/002fce8e93edd936d4b866118ae6d4c94e5d6744)
+- Added py37 [`d0ec862`](https://github.com/RhetTbull/osxphotos/commit/d0ec8620c721fe7576ab7d519a5eaac4d17a317e)
+
+#### [v0.31.0](https://github.com/RhetTbull/osxphotos/compare/v0.30.13...v0.31.0)
+
+> 27 July 2020
+
+- Initial FaceInfo support for Issue #21 [`6f29cda`](https://github.com/RhetTbull/osxphotos/commit/6f29cda99f1b8d94a95597c7046620cf21fecae4)
+- Updated Github Actions to run on PR [`9fc4f76`](https://github.com/RhetTbull/osxphotos/commit/9fc4f762193699dd45b586b51aa2d3066928aab1)
+
+#### [v0.30.13](https://github.com/RhetTbull/osxphotos/compare/v0.30.12...v0.30.13)
+
+> 23 July 2020
+
+- This reverts commit b7f4b739de978991def8ae2dca0f4e4b2881f56d, reversing [`#191`](https://github.com/RhetTbull/osxphotos/pull/191)
+- Fix findfiles not to fail on missing/invalid dir [`#192`](https://github.com/RhetTbull/osxphotos/pull/192)
+- Revert "Fix FileExistsError when filename differs only in case and export-as-hardlink (Bug#133)" [`#191`](https://github.com/RhetTbull/osxphotos/pull/191)
+- Fix FileExistsError when filename differs only in case and export-as-hardlink (Bug#133) [`#190`](https://github.com/RhetTbull/osxphotos/pull/190)
+- Fix findfiles not to fail on missing/invalid dir [`8c10b61`](https://github.com/RhetTbull/osxphotos/commit/8c10b61e90abbcfdff472bad4bb760558c7b850c)
+- Revert "Fix FileExistsError when filename differs only in case and export-as-hardlink (Bug#133)" [`f8e62d8`](https://github.com/RhetTbull/osxphotos/commit/f8e62d8f5ed26814f02383426237fd4c99a7ad04)
+- Fix FileExistsError when filename differs only in case and export-as-hardlink [`d52b387`](https://github.com/RhetTbull/osxphotos/commit/d52b387a294e68ebf0580a202ea70b97205560ef)
+- Version bump for bug fix [`cf4dca1`](https://github.com/RhetTbull/osxphotos/commit/cf4dca10c02d5f3f6132ab1572a698379b667e48)
+
+#### [v0.30.12](https://github.com/RhetTbull/osxphotos/compare/v0.30.10...v0.30.12)
+
+> 18 July 2020
+
+- Implemented PersonInfo, closes #181 [`#181`](https://github.com/RhetTbull/osxphotos/issues/181)
+- Updated dependencies, now supports py36, py37, py38 [`6688d1f`](https://github.com/RhetTbull/osxphotos/commit/6688d1ff6491f2e7e155946b265ef8b5d8929441)
+- Update README.md [`3526881`](https://github.com/RhetTbull/osxphotos/commit/3526881ec872cc009b0d8936f366afcfff166d42)
+
+#### [v0.30.10](https://github.com/RhetTbull/osxphotos/compare/v0.30.9...v0.30.10)
+
+> 6 July 2020
+
+- Bug fix for empty albums [`1ef518c`](https://github.com/RhetTbull/osxphotos/commit/1ef518cc3e9efbe9d4c16aa3d36c6dc6db86798e)
+
+#### [v0.30.9](https://github.com/RhetTbull/osxphotos/compare/v0.30.7...v0.30.9)
+
+> 6 July 2020
+
+- Refactored person processing to enable implementation of #181 [`fcff8ec`](https://github.com/RhetTbull/osxphotos/commit/fcff8ec5f8286b28e7d8559b40b5808a7b59cc15)
+- AlbumInfo.photos now returns photos in album sort order [`9d820a0`](https://github.com/RhetTbull/osxphotos/commit/9d820a0557944340d0c664a6c3497d138c6100d5)
+
+#### [v0.30.7](https://github.com/RhetTbull/osxphotos/compare/v0.30.6...v0.30.7)
+
+> 4 July 2020
+
+- Bug fix for keywords, persons in deleted photos [`df75a05`](https://github.com/RhetTbull/osxphotos/commit/df75a05645a88b31daa411f960d99ade71efc908)
+
+#### [v0.30.6](https://github.com/RhetTbull/osxphotos/compare/v0.30.5...v0.30.6)
+
+> 3 July 2020
+
+- Added height, width, orientation, filesize to json, str) [`8c3af0a`](https://github.com/RhetTbull/osxphotos/commit/8c3af0a4e4e49d9bbb33e809973d958334e44dca)
+
+#### [v0.30.5](https://github.com/RhetTbull/osxphotos/compare/v0.30.4...v0.30.5)
+
+> 3 July 2020
+
+- Added height, width, orientation, filesize, closes #163 [`#163`](https://github.com/RhetTbull/osxphotos/issues/163)
+
+#### [v0.30.4](https://github.com/RhetTbull/osxphotos/compare/v0.30.3...v0.30.4)
+
+> 3 July 2020
+
+- Added GPS location to XMP sidecar, closes #175 [`#175`](https://github.com/RhetTbull/osxphotos/issues/175)
+- Updated README.md [`7806e05`](https://github.com/RhetTbull/osxphotos/commit/7806e05673775ded231e65f53f3a1d5095a4b4e1)
+
+#### [v0.30.3](https://github.com/RhetTbull/osxphotos/compare/v0.30.2...v0.30.3)
+
+> 29 June 2020
+
+- Added --description-template to CLI, closes #166 [`#166`](https://github.com/RhetTbull/osxphotos/issues/166)
+- Added expand_inplace to PhotoTemplate.render [`ff03287`](https://github.com/RhetTbull/osxphotos/commit/ff0328785f3ea14b1c8ae2b7d1a9b07e8aef0777)
+- Updated README.md [`5950707`](https://github.com/RhetTbull/osxphotos/commit/59507077bafe39a17bc23babe6d6c52e1f502a53)
+
+#### [v0.30.2](https://github.com/RhetTbull/osxphotos/compare/v0.30.1...v0.30.2)
+
+> 28 June 2020
+
+- Added --deleted, --deleted-only to CLI, closes #179 [`#179`](https://github.com/RhetTbull/osxphotos/issues/179)
+
+#### [v0.30.1](https://github.com/RhetTbull/osxphotos/compare/v0.30.0...v0.30.1)
+
+> 27 June 2020
+
+- Changed default to PhotosDB.photos(movies=True), closes #177 [`#177`](https://github.com/RhetTbull/osxphotos/issues/177)
+
+#### [v0.30.0](https://github.com/RhetTbull/osxphotos/compare/v0.29.30...v0.30.0)
+
+> 27 June 2020
+
+- added intrash support for issue #179 [`185483e`](https://github.com/RhetTbull/osxphotos/commit/185483e1aa9ed107402bfb178f264417e6926b46)
+- Removed pdf filter on process_database_4 [`c1d1204`](https://github.com/RhetTbull/osxphotos/commit/c1d12047bde84740b96c8531110e7b2d2fe41f2e)
+
+#### [v0.29.30](https://github.com/RhetTbull/osxphotos/compare/v0.29.29...v0.29.30)
+
+> 24 June 2020
+
+- Added test for issue #178 [`46c87ee`](https://github.com/RhetTbull/osxphotos/commit/46c87eeed56d5765317dec4992d2e16323c711ad)
+- Additional fix for issue #178 [`fd4c990`](https://github.com/RhetTbull/osxphotos/commit/fd4c99032dbbedd6325aabacb0bc800b24ede413)
+
+#### [v0.29.29](https://github.com/RhetTbull/osxphotos/compare/v0.29.28...v0.29.29)
+
+> 23 June 2020
+
+- version bump [`d6fee89`](https://github.com/RhetTbull/osxphotos/commit/d6fee89fd9dd07c4788562ed551d0a3f2b5d697d)
+- Bug fix for issue #178 [`b8618cf`](https://github.com/RhetTbull/osxphotos/commit/b8618cf272efc174b7fa872f233b561bd9e7243e)
+
+#### [v0.29.28](https://github.com/RhetTbull/osxphotos/compare/v0.29.26...v0.29.28)
+
+> 22 June 2020
+
+- Closes #174 [`#174`](https://github.com/RhetTbull/osxphotos/issues/174)
+- Added today to template system, closes #167 [`#167`](https://github.com/RhetTbull/osxphotos/issues/167)
+- Minor refactoring in photoinfo.py [`a8e996e`](https://github.com/RhetTbull/osxphotos/commit/a8e996e66072e94de93fd4ea78a456bc61831f52)
+
+#### [v0.29.26](https://github.com/RhetTbull/osxphotos/compare/v0.29.25...v0.29.26)
+
+> 21 June 2020
+
+- Bug fix for issue #172 [`1ebf995`](https://github.com/RhetTbull/osxphotos/commit/1ebf99583397617f0d3a234c898beae1c14f5a63)
+
+#### [v0.29.25](https://github.com/RhetTbull/osxphotos/compare/v0.29.24...v0.29.25)
+
+> 21 June 2020
+
+- More PhotoInfo.albums refactoring, closes #169 [`#169`](https://github.com/RhetTbull/osxphotos/issues/169)
+
+#### [v0.29.24](https://github.com/RhetTbull/osxphotos/compare/v0.29.23...v0.29.24)
+
+> 21 June 2020
+
+- Refactored album code in photosdb to fix issue #169 [`cfabd0d`](https://github.com/RhetTbull/osxphotos/commit/cfabd0dbead62c8ab6a774899239e5da5bfe1203)
+
+#### [v0.29.23](https://github.com/RhetTbull/osxphotos/compare/v0.29.22...v0.29.23)
+
+> 20 June 2020
+
+- Fixed PhotoInfo.albums, album_info for issue #169 [`1212fad`](https://github.com/RhetTbull/osxphotos/commit/1212fad4adde0b4c6b2887392eed829d8d96d61d)
+
+#### [v0.29.22](https://github.com/RhetTbull/osxphotos/compare/v0.29.19...v0.29.22)
+
+> 19 June 2020
+
+- Don't raise KeyError when SystemLibraryPath is absent [`#168`](https://github.com/RhetTbull/osxphotos/pull/168)
+- Added check for export db in directory branch, closes #164 [`#164`](https://github.com/RhetTbull/osxphotos/issues/164)
+- Added OSXPhotosDB.get_db_connection() [`43d28e7`](https://github.com/RhetTbull/osxphotos/commit/43d28e78f394fa33f8d88f64b56b7dc7258cd454)
+- Added show() to photos_repl.py [`e98c3fe`](https://github.com/RhetTbull/osxphotos/commit/e98c3fe42912ac16d13675bf14154981089d41ea)
+- Fixed get_last_library_path and get_system_library_path to not raise KeyError [`5a83218`](https://github.com/RhetTbull/osxphotos/commit/5a832181f73e082927c80864f2063e554906b06b)
+- Don't raise KeyError when SystemLibraryPath is absent [`1fd0f96`](https://github.com/RhetTbull/osxphotos/commit/1fd0f96b14f0bc38e47bddb4cae12e19406324fb)
+
+#### [v0.29.19](https://github.com/RhetTbull/osxphotos/compare/v0.29.18...v0.29.19)
+
+> 14 June 2020
+
+- Added computed aesthetic scores, closes #141, closes #122 [`#141`](https://github.com/RhetTbull/osxphotos/issues/141) [`#122`](https://github.com/RhetTbull/osxphotos/issues/122)
+
+#### [v0.29.18](https://github.com/RhetTbull/osxphotos/compare/v0.29.17...v0.29.18)
+
+> 14 June 2020
+
+- Added --label to CLI, closes #157 [`#157`](https://github.com/RhetTbull/osxphotos/issues/157)
+
+#### [v0.29.17](https://github.com/RhetTbull/osxphotos/compare/v0.29.16...v0.29.17)
+
+> 13 June 2020
+
+- Extende --ignore-case to --person, --keyword, --album, closes #162 [`#162`](https://github.com/RhetTbull/osxphotos/issues/162)
+- Updated README.md to document template system [`0004250`](https://github.com/RhetTbull/osxphotos/commit/0004250e74eacc19f7986742712225116530a67e)
+
+#### [v0.29.16](https://github.com/RhetTbull/osxphotos/compare/v0.29.14...v0.29.16)
+
+> 13 June 2020
+
+- Added hour, min, sec, strftime templates, closes #158 [`#158`](https://github.com/RhetTbull/osxphotos/issues/158)
+- Added hour, min, sec to template system, issue #158 [`5387f8e`](https://github.com/RhetTbull/osxphotos/commit/5387f8e2f970ff7fa1967ccad87b45a4f7e50d32)
+
 #### [v0.29.14](https://github.com/RhetTbull/osxphotos/compare/v0.29.13...v0.29.14)
 
 > 13 June 2020
@@ -17,22 +370,20 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 7 June 2020
 
 - Added hidden debug-dump command to CLI [`7cd7b51`](https://github.com/RhetTbull/osxphotos/commit/7cd7b5159845fce15d50a7bfc0ac50d122bee527)
-- Updated CHANGELOG.md [`801dc62`](https://github.com/RhetTbull/osxphotos/commit/801dc62c4b7e24e1b92965ef6348113c440b1f9b)
 
 #### [v0.29.12](https://github.com/RhetTbull/osxphotos/compare/v0.29.9...v0.29.12)
 
 > 7 June 2020
 
+- Fix for bug in handling of deleted albums to address issue #156 [`72f034e`](https://github.com/RhetTbull/osxphotos/commit/72f034ef85010544a158d8301b898b5d0d865b05)
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`cb993f2`](https://github.com/RhetTbull/osxphotos/commit/cb993f2e5e2df7e0a15b3b2fdb92b65a8de56974)
 - Refactoring with sourceryAI [`5c7a0c3`](https://github.com/RhetTbull/osxphotos/commit/5c7a0c3a246cd5fec329b4fd4979d2b77352f916)
-- Partial fix for #155 [`2271d89`](https://github.com/RhetTbull/osxphotos/commit/2271d8935507ecc27e6227b11b4796f2f4d2f10d)
-- Partial fix for #155 [`62d096b`](https://github.com/RhetTbull/osxphotos/commit/62d096b5a1a7e960195ec5c48fc9cffbebf2c735)
 
 #### [v0.29.9](https://github.com/RhetTbull/osxphotos/compare/v0.29.8...v0.29.9)
 
 > 31 May 2020
 
 - Added --filename to CLI, closes #89 [`#89`](https://github.com/RhetTbull/osxphotos/issues/89)
-- Updated CHANGELOG.md [`d47fd46`](https://github.com/RhetTbull/osxphotos/commit/d47fd46a21881bea86d1bc624c6027e2cbe08d9c)
 
 #### [v0.29.8](https://github.com/RhetTbull/osxphotos/compare/v0.29.5...v0.29.8)
 
@@ -54,7 +405,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Added --dry-run option to CLI export, closes #91 [`#91`](https://github.com/RhetTbull/osxphotos/issues/91)
 - added created.dd and modified.dd to template system, closes #135 [`#135`](https://github.com/RhetTbull/osxphotos/issues/135)
 - Catch exception in folder processing to address #148 [`46fdc94`](https://github.com/RhetTbull/osxphotos/commit/46fdc94398c80b157048649434c7312074ce5c58)
-- Updated CHANGELOG.md [`af750dd`](https://github.com/RhetTbull/osxphotos/commit/af750dd2e392be1a7163cf32497526405665ea70)
+- added created.dow (day of week) to template [`8df6d2c`](https://github.com/RhetTbull/osxphotos/commit/8df6d2c707caf4eb35696888282365a128b69569)
 - Added test for DateTimeFormatter.dow [`09c7d18`](https://github.com/RhetTbull/osxphotos/commit/09c7d18901b61669d8b9242babd82eba6987c89a)
 
 #### [v0.29.2](https://github.com/RhetTbull/osxphotos/compare/v0.29.1...v0.29.2)
@@ -68,7 +419,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 23 May 2020
 
 - Catch illegal timestamp value [`#146`](https://github.com/RhetTbull/osxphotos/pull/146)
-- Updated CHANGELOG.md [`1450b3c`](https://github.com/RhetTbull/osxphotos/commit/1450b3ccace326fe1c0ed810a1b40e781709acb3)
+- Catch illegal timestamp value [`441de71`](https://github.com/RhetTbull/osxphotos/commit/441de711dc664b244d599c81e3dd1bcd9b2e55a0)
 
 #### [v0.29.0](https://github.com/RhetTbull/osxphotos/compare/v0.28.19...v0.29.0)
 
@@ -78,8 +429,8 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Added --update to CLI export; reference issue #100 [`b1171e9`](https://github.com/RhetTbull/osxphotos/commit/b1171e96cc06362555725995bb311317eb163e49)
 - Added as_dict to PlaceInfo [`8c4fe40`](https://github.com/RhetTbull/osxphotos/commit/8c4fe40aa6850f166e526cffaa088550884399af)
 - Updated README.md [`11d368a`](https://github.com/RhetTbull/osxphotos/commit/11d368a69cbe67e909e64b020f0334fc09dd3ac4)
-- Updated CHANGELOG.md [`cafa483`](https://github.com/RhetTbull/osxphotos/commit/cafa483cfc228c651a03d3361d6d48a35deab1e8)
 - version bump [`c06c230`](https://github.com/RhetTbull/osxphotos/commit/c06c230a469754691d11fff1034fb02daeeba649)
+- Test library update [`f416418`](https://github.com/RhetTbull/osxphotos/commit/f416418546a12bc6c1bda13f6b712758584d06dc)
 
 #### [v0.28.19](https://github.com/RhetTbull/osxphotos/compare/v0.28.18...v0.28.19)
 
@@ -88,16 +439,15 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Added label and label_normalized to template system, closes #130 [`#130`](https://github.com/RhetTbull/osxphotos/issues/130)
 - Revert "test library updates" [`48e9c32`](https://github.com/RhetTbull/osxphotos/commit/48e9c32add549e66c3ef8c65f8821f5033b55b11)
 - test library updates [`d125854`](https://github.com/RhetTbull/osxphotos/commit/d125854f2a04e37747af3e0796370a565c1c9bd0)
-- Updated CHANGELOG.md [`e228cfa`](https://github.com/RhetTbull/osxphotos/commit/e228cfab746055c8d6df428aebe0ed001fb6d4d0)
 - version bump [`bd9d5a2`](https://github.com/RhetTbull/osxphotos/commit/bd9d5a26f3bfcbb33896a139fa86cdab46768103)
 - Update README.md [`85760dc`](https://github.com/RhetTbull/osxphotos/commit/85760dc4fe2274d826ed80494fd4e66866398609)
+- Update README.md [`be07f90`](https://github.com/RhetTbull/osxphotos/commit/be07f90e5a8179e452730ea654e4c9627b1f6ebc)
 
 #### [v0.28.18](https://github.com/RhetTbull/osxphotos/compare/v0.28.17...v0.28.18)
 
 > 14 May 2020
 
 - Implemented PhotoInfo.exiftool [`a80dee4`](https://github.com/RhetTbull/osxphotos/commit/a80dee401c7eb959f6ad6d93a3272657ed28f521)
-- Updated CHANGELOG.md [`e67fce2`](https://github.com/RhetTbull/osxphotos/commit/e67fce28714cf4065b64202bb3b149ba5bec5be4)
 
 #### [v0.28.17](https://github.com/RhetTbull/osxphotos/compare/v0.28.15...v0.28.17)
 
@@ -114,8 +464,8 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - added --export-as-hardlink option [`#126`](https://github.com/RhetTbull/osxphotos/pull/126)
 - Added test for folder_names on 10.15.4, closes #119 [`#119`](https://github.com/RhetTbull/osxphotos/issues/119)
 - Refactored photosdb and photoinfo to add SearchInfo and labels [`98b3f63`](https://github.com/RhetTbull/osxphotos/commit/98b3f63a92aa2105f8fa97af992fc6fe2d78b973)
+- added --export-as-hardlink option [`5eb0876`](https://github.com/RhetTbull/osxphotos/commit/5eb0876e331beb020431bb037dee75fb7ae61c85)
 - Added additional test for --export-as-hardlink [`57315d4`](https://github.com/RhetTbull/osxphotos/commit/57315d44497fde977956f76f667470208f11aa2d)
-- added CHANGELOG.md [`00e1661`](https://github.com/RhetTbull/osxphotos/commit/00e16611fc86c05fb090d036084db9eb42444071)
 - Updated a couple of tests to use pytest-mock [`397db0d`](https://github.com/RhetTbull/osxphotos/commit/397db0d72fb218669a9ecbff134fa9b392a14661)
 - added test for export using hardlinks, fixed a test that failed if users locale settings were different to en_US [`b0ec6c6`](https://github.com/RhetTbull/osxphotos/commit/b0ec6c6b36d8cfe05723d47b210d9d7c5aabdfe5)
 
@@ -139,7 +489,6 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 28 April 2020
 
 - Fixed implementation of use_albums_as_keywords and use_persons_as_keywords, closes #115 [`#115`](https://github.com/RhetTbull/osxphotos/issues/115)
-- Updated CHANGELOG.md [`072a8d7`](https://github.com/RhetTbull/osxphotos/commit/072a8d795e5e15fa8ca8d8872aecf4cddd7837f7)
 - Update README.md [`5cc98c3`](https://github.com/RhetTbull/osxphotos/commit/5cc98c338bcc19fd05bf293eb3afe24c07c8b380)
 - Updated README.md [`a800711`](https://github.com/RhetTbull/osxphotos/commit/a80071111f810a1d7d6e2d735839e85499091ea4)
 - Update README.md [`1c9d4f2`](https://github.com/RhetTbull/osxphotos/commit/1c9d4f282beea2ac12273c8d0f9453bad1255c2c)
@@ -150,7 +499,6 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Added --album-keyword and --person-keyword to CLI, closes #61 [`#61`](https://github.com/RhetTbull/osxphotos/issues/61)
 - Updated test libraries [`54d5d4b`](https://github.com/RhetTbull/osxphotos/commit/54d5d4b7ba99204f58e723231309ab6e306be28c)
-- Updated CHANGELOG.md [`38137a1`](https://github.com/RhetTbull/osxphotos/commit/38137a1351cdb7ab72393ea03828933dac0b76b0)
 - Updated tests/README.md [`56a0006`](https://github.com/RhetTbull/osxphotos/commit/56a000609f2f08d0f8800fec49cada2980c3bb9d)
 
 #### [v0.28.6](https://github.com/RhetTbull/osxphotos/compare/v0.28.5...v0.28.6)
@@ -158,7 +506,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 26 April 2020
 
 - Fixed locale bug in templates, closes #113 [`#113`](https://github.com/RhetTbull/osxphotos/issues/113)
-- Updated CHANGELOG.md [`81d4e39`](https://github.com/RhetTbull/osxphotos/commit/81d4e392c39f0fe6f967a447c7d0c970bf224032)
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`4b29a2e`](https://github.com/RhetTbull/osxphotos/commit/4b29a2e05fd1dac821d80781ae01a148d3d9c523)
 - Updated test to avoid issue with GitHub workflow [`9be0f84`](https://github.com/RhetTbull/osxphotos/commit/9be0f849b73061d053d30274ff3295b79c88f0b6)
 - Update pythonpackage.yml to remove older pythons [`ccb5f25`](https://github.com/RhetTbull/osxphotos/commit/ccb5f252d14e9335ae04a2e338a6d527b80c9a93)
 
@@ -172,7 +520,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Updated setup.py to resolve issue with bpylist2 on python < 3.8 [`8e4b88a`](https://github.com/RhetTbull/osxphotos/commit/8e4b88ad1fc18438f941e045bfc8aeac878914f9)
 - Added cli.py for use with pyinstaller [`cf28cb6`](https://github.com/RhetTbull/osxphotos/commit/cf28cb6452de17f2ef8d80435386e8d5a1aabd34)
 - added raw_is_original handling [`a337e79`](https://github.com/RhetTbull/osxphotos/commit/a337e79e13802b4824c2f088ce9db1c027d6f3c5)
-- Updated CHANGELOG.md [`22f1e8f`](https://github.com/RhetTbull/osxphotos/commit/22f1e8f2a6478e0576f6bff53e348aad8680ae69)
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`1c8eb76`](https://github.com/RhetTbull/osxphotos/commit/1c8eb764f53c3cc8b541667c858e462793ad8d1f)
 
 #### [0.28.2](https://github.com/RhetTbull/osxphotos/compare/v0.28.1...0.28.2)
 
@@ -180,7 +528,6 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Added folder support for Photos <= 4, closes #93 [`#93`](https://github.com/RhetTbull/osxphotos/issues/93)
 - cleaned up SQL statements in _process_database4 [`6f28171`](https://github.com/RhetTbull/osxphotos/commit/6f281711e2001a63ffad076d7b9835272d5d09da)
-- Updated CHANGELOG.md [`1fa9583`](https://github.com/RhetTbull/osxphotos/commit/1fa9583ea689d54d2613a064f1ade25bcdfbf043)
 - Fixed suffix check on export to be case insensitive [`4b30b3b`](https://github.com/RhetTbull/osxphotos/commit/4b30b3b4260e2c7409e18825e5b626efe646db16)
 - test library update [`3bac106`](https://github.com/RhetTbull/osxphotos/commit/3bac106eb7a180e9e39643a89087d92bf2a437d0)
 
@@ -198,7 +545,6 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Added {folder_album} to template and --folder to CLI [`b7c7b9f`](https://github.com/RhetTbull/osxphotos/commit/b7c7b9f0664e69c743bdd8a228ad2936cf6b7600)
 - Test library update [`21e7020`](https://github.com/RhetTbull/osxphotos/commit/21e7020fec406b0f3926d7adc8a1451bfe77e75a)
-- Updated CHANGELOG.md [`952741d`](https://github.com/RhetTbull/osxphotos/commit/952741d488d2fbbaf8a0c1d3781ad7c4205c068f)
 
 #### [v0.27.3](https://github.com/RhetTbull/osxphotos/compare/v0.27.1...v0.27.3)
 
@@ -212,7 +558,6 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 12 April 2020
 
 - Changed AlbumInfo and FolderInfo interface to maintain backwards compatibility with PhotosDB.albums [`e09f0b4`](https://github.com/RhetTbull/osxphotos/commit/e09f0b40f1671d70ee399cdc519492b04fac8adc)
-- Updated CHANGELOG.md [`b749681`](https://github.com/RhetTbull/osxphotos/commit/b749681c6d2545eacf653ab1b2a5d1384e3123eb)
 
 #### [v0.27.0](https://github.com/RhetTbull/osxphotos/compare/v0.26.1...v0.27.0)
 
@@ -221,8 +566,8 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Update README.md [`#95`](https://github.com/RhetTbull/osxphotos/pull/95)
 - Added tests and README for AlbumInfo and FolderInfo [`d6a22b7`](https://github.com/RhetTbull/osxphotos/commit/d6a22b765ab17f6ef1ba8c50b77946f090979968)
 - Added albuminfo.py for AlbumInfo and FolderInfo classes [`9636572`](https://github.com/RhetTbull/osxphotos/commit/96365728c2ff42abfb6828872ffac53b4c3c8024)
-- Updated CHANGELOG.md [`cde56e9`](https://github.com/RhetTbull/osxphotos/commit/cde56e9d13baf3098ec85839cf1aaa33b4915ac9)
 - Update README.md TOC [`8544667`](https://github.com/RhetTbull/osxphotos/commit/8544667c729ea0d7fe39671d909e09cda519e250)
+- Update README.md [`1aa3838`](https://github.com/RhetTbull/osxphotos/commit/1aa3838c3866a18084ffe822de02df0eda464d71)
 
 #### [v0.26.1](https://github.com/RhetTbull/osxphotos/compare/v0.26.0...v0.26.1)
 
@@ -230,7 +575,6 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Bug fix for PhotosDB.photos() query [`1c9da5e`](https://github.com/RhetTbull/osxphotos/commit/1c9da5ed6ffa21f0577906b65b7da08951725d1f)
 - Updated test library [`d74f7f4`](https://github.com/RhetTbull/osxphotos/commit/d74f7f499bf59f37ec81cfa9d49cbbf3aafb5961)
-- Updated CHANGELOG.md [`c85bb02`](https://github.com/RhetTbull/osxphotos/commit/c85bb023042e072d6688060eb259156c2fa579b9)
 
 #### [v0.26.0](https://github.com/RhetTbull/osxphotos/compare/v0.25.1...v0.26.0)
 
@@ -238,7 +582,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Added test for 10.15.4 [`1820715`](https://github.com/RhetTbull/osxphotos/commit/182071584904d001a9b199eef5febfb79e00696e)
 - Changed PhotosDB albums interface as prep for adding folders [`3e50626`](https://github.com/RhetTbull/osxphotos/commit/3e5062684ab6d706d91d4abeb4e3b0ca47867b70)
-- Updated CHANGELOG.md [`a6ca3f4`](https://github.com/RhetTbull/osxphotos/commit/a6ca3f453ce0fae4e8d13c7c256ed69a16d2e3f2)
+- Update README.md [`626e460`](https://github.com/RhetTbull/osxphotos/commit/626e460aabb97b30af87cea2ec4f93e5fb925bec)
 
 #### [v0.25.1](https://github.com/RhetTbull/osxphotos/compare/v0.25.0...v0.25.1)
 
@@ -257,7 +601,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Added {album}, {keyword}, and {person} to template system [`507c4a3`](https://github.com/RhetTbull/osxphotos/commit/507c4a374014f999ca19789bce0df0c14332e021)
 - Added places command to CLI [`fd5e748`](https://github.com/RhetTbull/osxphotos/commit/fd5e748dca759ea1c3a7329d447f363afe8418b7)
 - Updated export example [`01cd7fe`](https://github.com/RhetTbull/osxphotos/commit/01cd7fed6d7fc0c61c171a05319c211eb0a9f7c1)
-- Updated CHANGELOG.md [`daea30f`](https://github.com/RhetTbull/osxphotos/commit/daea30f1626a208209ab6854cbd3b12f4b0a3405)
+- Fixed typo in help text [`c02953e`](https://github.com/RhetTbull/osxphotos/commit/c02953ef5fe1aee219e0557bfd8c3322f1900a81)
 
 #### [v0.24.2](https://github.com/RhetTbull/osxphotos/compare/v0.24.1...v0.24.2)
 
@@ -278,8 +622,8 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 22 March 2020
 
 - Added export_by_album.py to examples [`908fead`](https://github.com/RhetTbull/osxphotos/commit/908fead8a2fbcef3b4a387f34d83d88c507c5939)
-- Updated CHANGELOG.md [`072e894`](https://github.com/RhetTbull/osxphotos/commit/072e894e56c4dfe5522d073b202933fed0204ef5)
 - Updated pathvalidate calls [`d066435`](https://github.com/RhetTbull/osxphotos/commit/d066435e3df4062be6a0a3d5fa7308f293e764d5)
+- Updated example [`8f0307f`](https://github.com/RhetTbull/osxphotos/commit/8f0307fc24345ca0e87017ac76791c9bbe8db25e)
 
 #### [v0.23.3](https://github.com/RhetTbull/osxphotos/compare/v0.23.1...v0.23.3)
 
@@ -294,15 +638,15 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 21 March 2020
 
 - Fixed requirements.txt for bplist2 [`cda5f44`](https://github.com/RhetTbull/osxphotos/commit/cda5f446933ea2272409d1f153e2a7811626ada6)
-- Updated CHANGELOG.md [`b8da976`](https://github.com/RhetTbull/osxphotos/commit/b8da9765b8949eb90852d249c2877eeb1806d987)
 - Updated requirements.txt [`9da7ad6`](https://github.com/RhetTbull/osxphotos/commit/9da7ad6dcc021fdafe358d74e1c52f69dc49ade8)
+- still trying to debug github actions fail [`960487f`](https://github.com/RhetTbull/osxphotos/commit/960487f2961f97f6b24d253472dcedf74dfc7797)
 
 #### [v0.23.0](https://github.com/RhetTbull/osxphotos/compare/v0.22.23...v0.23.0)
 
 > 21 March 2020
 
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`21547a8`](https://github.com/RhetTbull/osxphotos/commit/21547a8eaad117b11bc5e4dddf95436a8244e9ba)
 - Added PhotoInfo.place for reverse geolocation data [`b338b34`](https://github.com/RhetTbull/osxphotos/commit/b338b34d5055a7621e4ebe4fbbae12227d77af6d)
-- Updated CHANGELOG.md [`816b98e`](https://github.com/RhetTbull/osxphotos/commit/816b98e617c30d0bdb51bc2413f9915742c8592e)
 - Update pythonpackage.yml [`92e5bdd`](https://github.com/RhetTbull/osxphotos/commit/92e5bdd2e986e5de2a710abf60ba0dc99c6a6730)
 
 #### [v0.22.23](https://github.com/RhetTbull/osxphotos/compare/v0.22.21...v0.22.23)
@@ -318,7 +662,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Working on export edited bug for issue #78 [`8542e1a`](https://github.com/RhetTbull/osxphotos/commit/8542e1a97f6b640f287b37af9e50fd05f964ec4d)
 - Fixed download-missing to only download when actually missing [`dd20b8d`](https://github.com/RhetTbull/osxphotos/commit/dd20b8d8ac3b16d3b72a26b97dcc620b11e3a7c0)
-- Updated CHANGELOG.md [`cc9220e`](https://github.com/RhetTbull/osxphotos/commit/cc9220e0763816d784f2fd8377dfe14a99981622)
+- test library updates [`e99391a`](https://github.com/RhetTbull/osxphotos/commit/e99391a68e844adb63edde3efb921cffa3928aeb)
 
 #### [v0.22.17](https://github.com/RhetTbull/osxphotos/compare/v0.22.16...v0.22.17)
 
@@ -333,18 +677,17 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - removed activate from --download-missing-photos Applescript, closes #69 [`#69`](https://github.com/RhetTbull/osxphotos/issues/69)
 - Added media type specials to json and string output, closes #68 [`#68`](https://github.com/RhetTbull/osxphotos/issues/68)
+- Fixed bug in --download-missing related to burst images [`1f13ba8`](https://github.com/RhetTbull/osxphotos/commit/1f13ba837fe36ff4eeb48cca02f5312a88a0a765)
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`dc87194`](https://github.com/RhetTbull/osxphotos/commit/dc87194eec252461d0cc0891b9ede4157125e828)
 - Added query/export options for special media types [`2b7d84a`](https://github.com/RhetTbull/osxphotos/commit/2b7d84a4d103982ad874d875bafbc34d654d539a)
 - README.md update [`a27ce33`](https://github.com/RhetTbull/osxphotos/commit/a27ce33473df3260dfb7ed26e28295cbf87d1e78)
 - Test library updates [`2d7d0b8`](https://github.com/RhetTbull/osxphotos/commit/2d7d0b86e0008cae043e314937504f36ad882990)
-- Fixed bug in --download-missing related to burst images [`1f13ba8`](https://github.com/RhetTbull/osxphotos/commit/1f13ba837fe36ff4eeb48cca02f5312a88a0a765)
-- test library update [`acb6b9e`](https://github.com/RhetTbull/osxphotos/commit/acb6b9e72f7f6b8f4f1d64b46f270a4d3e984fef)
 
 #### [v0.22.13](https://github.com/RhetTbull/osxphotos/compare/v0.22.12...v0.22.13)
 
 > 8 March 2020
 
 - Added media type specials, closes #60 [`#60`](https://github.com/RhetTbull/osxphotos/issues/60)
-- Updated CHANGELOG.md [`08a9793`](https://github.com/RhetTbull/osxphotos/commit/08a9793651481e1984a4482794ffedd48e4367a2)
 - Updated README.md [`1f8fd6e`](https://github.com/RhetTbull/osxphotos/commit/1f8fd6e929cc0edd3dd2f222416454d26955bf2a)
 
 #### [v0.22.12](https://github.com/RhetTbull/osxphotos/compare/0.22.10...v0.22.12)
@@ -361,7 +704,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Fixed bug in --download-missing to fix issue #64 [`c654e3d`](https://github.com/RhetTbull/osxphotos/commit/c654e3dc61283382b37b6892dab1516ec517143a)
 - removed commented out code [`69addc3`](https://github.com/RhetTbull/osxphotos/commit/69addc34649f992c6a4a0e0e334754a72530f0ba)
-- Updated CHANGELOG.md [`1e013b6`](https://github.com/RhetTbull/osxphotos/commit/1e013b6802e49e26ec5a94eb702e841b2eb68395)
+- Cleaned up comments and unneeded test code [`e3c40bc`](https://github.com/RhetTbull/osxphotos/commit/e3c40bcbaaf3560d53091cf46ed851d90ff82cfa)
 
 #### [v0.22.9](https://github.com/RhetTbull/osxphotos/compare/v0.22.7...v0.22.9)
 
@@ -373,7 +716,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Test library updates [`6e563e2`](https://github.com/RhetTbull/osxphotos/commit/6e563e214c569ba7838f7464de9258c3bba5db23)
 - Removed _tmp_file code that's no longer needed [`27994c9`](https://github.com/RhetTbull/osxphotos/commit/27994c9fd372303833a5794f1de9815f425c762e)
 - Updated photos_repl.py [`fdf636a`](https://github.com/RhetTbull/osxphotos/commit/fdf636ac8864ebb2cc324b1f9d3c6c82ee3910f9)
-- Updated CHANGELOG.md [`f910124`](https://github.com/RhetTbull/osxphotos/commit/f910124fe1fbf75d44c09c79607374bf000733a1)
+- Added PhotosDB() behavior to open last library if no args passed but also added cautionary note to README [`46d3c7d`](https://github.com/RhetTbull/osxphotos/commit/46d3c7dbdaf848d5c340ce8a362ff296a36c552d)
 
 #### [v0.22.7](https://github.com/RhetTbull/osxphotos/compare/v0.22.4...v0.22.7)
 
@@ -387,7 +730,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Added date_modified to PhotoInfo [`67b0ae0`](https://github.com/RhetTbull/osxphotos/commit/67b0ae0bf679815372d415c3064e21d46a5b8718)
 - Added date_modified to PhotoInfo [`4d36b3b`](https://github.com/RhetTbull/osxphotos/commit/4d36b3b31f3e0e74d9d111b6b691771e19f94086)
 - Updated CLI options with more descriptive metavar names [`e79cb92`](https://github.com/RhetTbull/osxphotos/commit/e79cb92693758c984dc789d5fa5d2e87e381e921)
-- CLI now looks for photos library to use if non specified by user [`50b7e69`](https://github.com/RhetTbull/osxphotos/commit/50b7e6920a694aa45f478d1131868525c9147919)
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`898d3af`](https://github.com/RhetTbull/osxphotos/commit/898d3afc0892546ece6c3d675208dea216e20633)
 
 #### [v0.22.4](https://github.com/RhetTbull/osxphotos/compare/v0.22.0...v0.22.4)
 
@@ -399,7 +742,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Implement from_date and to_date in PhotosDB as well as query and export command. Some refactoring of CLI as well. [`cfa2b4a`](https://github.com/RhetTbull/osxphotos/commit/cfa2b4a828facf0aff5bc19f777457ad776c4a05)
 - Refactored _query.  Still hairy, but less so. [`b9dee49`](https://github.com/RhetTbull/osxphotos/commit/b9dee4995c6d89fadb3d2482374b7098f2ab5ed9)
 - Updated README.md [`0aff83f`](https://github.com/RhetTbull/osxphotos/commit/0aff83ff21c20e293c0b75bacf2863090a0fb725)
-- Started adding tests for CLI [`f0b18c3`](https://github.com/RhetTbull/osxphotos/commit/f0b18c3d29b2141d348be0495013c51c072c6251)
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`7150956`](https://github.com/RhetTbull/osxphotos/commit/7150956a488677d402a6d43443d04c4b11dc7be0)
 
 #### [v0.22.0](https://github.com/RhetTbull/osxphotos/compare/v0.21.5...v0.22.0)
 
@@ -407,7 +750,7 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 - Refactored PhotosDB and CLI to require explicity passing the database to avoid non-deterministic behavior when last database can't be found.  This may break existing code. [`ede56ff`](https://github.com/RhetTbull/osxphotos/commit/ede56ffc31cf98811b3d4d16e22406ac0eae0315)
 - Changed get_system_library_path to return None if could not get system library [`646ea4f`](https://github.com/RhetTbull/osxphotos/commit/646ea4f24ca1119b27280af1445e31adcd0690f0)
-- Updated CHANGELOG.md [`bd20388`](https://github.com/RhetTbull/osxphotos/commit/bd20388778dfa645277029601c63fc9835b7a406)
+- Fix to setup to specify versions of required packages [`de05323`](https://github.com/RhetTbull/osxphotos/commit/de05323a153fe49723b39e48b9038c1fb9535a72)
 
 #### [v0.21.5](https://github.com/RhetTbull/osxphotos/compare/v0.21.0...v0.21.5)
 
@@ -422,8 +765,8 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 > 4 January 2020
 
 - Added live photo support for both Photos 4 & 5 [`d5eaff0`](https://github.com/RhetTbull/osxphotos/commit/d5eaff02f2a29a9d105ab72e9a9aeffbc9a3425b)
-- Added support for burst photos; added export-bursts to CLI [`593983a`](https://github.com/RhetTbull/osxphotos/commit/593983a09940e67fb9347bf345cfd7289465fa0a)
 - Added live-photo option to CLI query and export [`6f6d37c`](https://github.com/RhetTbull/osxphotos/commit/6f6d37ceacf71a52a2c0216f0ad75afee244946a)
+- Initial support for live photos (Photos 5 only) [`1a89a18`](https://github.com/RhetTbull/osxphotos/commit/1a89a18a011a25616d7a18fb9bf1270b0b206fb4)
 
 #### [v0.20.0](https://github.com/RhetTbull/osxphotos/compare/v0.19.0...v0.20.0)
 
@@ -437,9 +780,9 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
 > 29 December 2019
 
+- Merge branch 'master' of https://github.com/RhetTbull/osxphotos [`51843fb`](https://github.com/RhetTbull/osxphotos/commit/51843fb46d6ce69456400271c97aa642466d5719)
 - Added support for movies for Photos 5; fixed bugs in ismissing and path [`6f4d129`](https://github.com/RhetTbull/osxphotos/commit/6f4d129f07046c4a34d3d6cf6854c8514a594781)
 - Added support for movies for Photos 5; fixed bugs in ismissing and path [`b030966`](https://github.com/RhetTbull/osxphotos/commit/b030966051af93be380ff967ac047bf566e5d817)
-- Initial support for movies [`dbe363e`](https://github.com/RhetTbull/osxphotos/commit/dbe363e4d754253a0405fb1df045677e8780d630)
 
 #### [v0.18.0](https://github.com/RhetTbull/osxphotos/compare/v0.15.1...v0.18.0)
 

examples/export_by_album.py

@@ -79,7 +79,7 @@ def export(export_path, default_album, library_path, edited):
                 exported = p.export(dest_dir, filename)
                 click.echo(f"Exported {filename} to {exported}")
         else:
-            click.echo(f"Skipping missing photo: {p.original_filename} in album {album}")
+            click.echo(f"Skipping missing photo: {p.original_filename}")
 
 
 if __name__ == "__main__":

examples/export_faces.py

@@ -0,0 +1,83 @@
+""" Export all photos that contain a detected face and draw rectangles around each face
+    photos with no persons/detected faces will not be export 
+
+    This shows how to use the FaceInfo class and is useful for validating that FaceInfo is
+    correctly handling faces.
+
+    To use this, you'll need to install Pillow:
+    python3 -m pip install Pillow
+"""
+
+import os
+
+import click
+from PIL import Image, ImageDraw
+
+import osxphotos
+
+
+@click.command()
+@click.argument("export-path", type=click.Path(exists=True))
+@click.option(
+    "--uuid",
+    metavar="UUID",
+    help="Limit export to optional UUID(s)",
+    required=False,
+    multiple=True,
+)
+@click.option(
+    "--library-path",
+    metavar="PATH",
+    help="Path to Photos library, default to last used library",
+    default=None,
+)
+def export(export_path, library_path, uuid):
+    """ export photos to export_path and draw faces """
+    library_path = os.path.expanduser(library_path) if library_path else None
+    if library_path is not None:
+        photosdb = osxphotos.PhotosDB(library_path)
+    else:
+        photosdb = osxphotos.PhotosDB()
+
+    photos = photosdb.photos(uuid=uuid) if uuid else photosdb.photos(movies=False)
+    for p in photos:
+        if p.person_info and not p.ismissing:
+            # has persons and not missing
+            if "heic" in p.filename.lower():
+                print(f"skipping heic image {p.filename}")
+                continue
+            print(f"exporting photo {p.original_filename}, uuid = {p.uuid}")
+            export = p.export(export_path, p.original_filename, edited=p.hasadjustments)
+            if export:
+                im = Image.open(export[0])
+                draw = ImageDraw.Draw(im)
+                for face in p.face_info:
+                    coords = face.face_rect()
+                    draw.rectangle(coords, width=3)
+                    draw.ellipse(get_circle_points(face.center, 3), width=1)
+                    draw.text(face.mouth, "M", fill=(255, 255, 255, 255))
+                    draw.text(face.left_eye, "L", fill=(255, 255, 255, 255))
+                    draw.text(face.right_eye, "R", fill=(255, 255, 255, 255))
+                im.save(export[0])
+            else:
+                print(f"no photos exported for {p.uuid}")
+
+
+def get_circle_points(xy, radius):
+    """ Returns tuples of (x0, y0), (x1, y1) for a circle centered at x, y with radius
+
+    Arguments:
+        xy: tuple of x, y coordinates
+        radius: radius of circle to draw
+
+    Returns:
+        [(x0, y0), (x1, y1)] for bounding box of circle centered at x, y
+    """
+    x, y = xy
+    x0, y0 = x - radius, y - radius
+    x1, y1 = x + radius, y + radius
+    return [(x0, y0), (x1, y1)]
+
+
+if __name__ == "__main__":
+    export()  # pylint: disable=no-value-for-parameter

examples/force_download.py

@@ -0,0 +1,42 @@
+""" use osxphotos to force the download of photos from iCloud
+    downloads images to a temporary directory then deletes them
+    resulting in the photo being downloaded to Photos library
+"""
+
+import os
+import sys
+import tempfile
+
+import osxphotos
+
+
+def main():
+    photosdb = osxphotos.PhotosDB()
+    tempdir = tempfile.TemporaryDirectory()
+    photos = photosdb.photos()
+    downloaded = 0
+    missing = [photo for photo in photos if photo.ismissing and not photo.shared]
+
+    if not missing:
+        print(f"Did not find any missing photos to download")
+        sys.exit(0)
+
+    print(f"Downloading {len(missing)} photos")
+    for photo in missing:
+        if photo.ismissing:
+            print(f"Downloading photo {photo.original_filename}")
+            downloaded += 1
+            exported = photo.export(tempdir.name, use_photos_export=True, timeout=300)
+            if photo.hasadjustments:
+                exported.extend(
+                    photo.export(tempdir.name, use_photos_export=True, edited=True, timeout=300)
+                )
+            for filename in exported:
+                print(f"Removing temporary file {filename}")
+                os.unlink(filename)
+    print(f"Downloaded {downloaded} photos")
+    tempdir.cleanup()
+
+
+if __name__ == "__main__":
+    main()

examples/get_shared_photo_comments.py

@@ -0,0 +1,156 @@
+""" get shared comments associated with a photo """
+
+import datetime
+import sys
+from dataclasses import dataclass
+
+import osxphotos
+from osxphotos._constants import TIME_DELTA
+
+
+@dataclass
+class Comment:
+    """ Class for shared photo comments """
+
+    uuid: str
+    sort_fok: int
+    datetime: datetime.datetime
+    user: str
+    ismine: bool
+    text: str
+
+
+@dataclass
+class Like:
+    """ Class for shared photo likes """
+
+    uuid: str
+    sort_fok: int
+    datetime: datetime.datetime
+    user: str
+    ismine: bool
+
+
+def get_shared_person_info(photosdb, hashed_person_id):
+    """ returns tuple of (first name, last name, full name) 
+        for person invited to shared album with 
+        ZINVITEEHASHEDPERSONID = hashed_person_id
+        
+    Args:
+        photosdb: a osxphotos.PhotosDB object
+        hashed_person_id: str, value of ZINVITEEHASHEDPERSONID to lookup
+    """
+
+    conn, _ = photosdb.get_db_connection()
+    results = conn.execute(
+        """
+        SELECT
+        ZINVITEEHASHEDPERSONID,
+        ZINVITEEFIRSTNAME,
+        ZINVITEELASTNAME,
+        ZINVITEEFULLNAME
+        FROM
+        ZCLOUDSHAREDALBUMINVITATIONRECORD
+        WHERE
+        ZINVITEEHASHEDPERSONID = ?
+        LIMIT 1
+        """,
+        ([hashed_person_id]),
+    ).fetchall()
+
+    if results:
+        row = results[0]
+        return (row[1], row[2], row[3])
+    else:
+        return (None, None, None)
+
+
+def get_comments(photosdb, uuid):
+    """ return comments and likes, if any, for photo with uuid
+
+    Args:
+        photosdb: a osxphotos.PhotosDB object
+        uuid: uuid of the photo
+    
+    Returns:
+        tuple of (list of comments as Comment objects or [] if no comments, list of likes as Like objects or [] if no likes)
+    """
+    conn, _ = photosdb.get_db_connection()
+
+    results = conn.execute(
+        """
+        SELECT 
+        ZGENERICASSET.ZUUID, --0: UUID of the photo
+        ZCLOUDSHAREDCOMMENT.ZISLIKE, --1: comment is actually a "like"
+        ZCLOUDSHAREDCOMMENT.Z_FOK_COMMENTEDASSET, --2: sort order for comments on a photo
+        ZCLOUDSHAREDCOMMENT.ZCOMMENTDATE, --3: date of comment
+        ZCLOUDSHAREDCOMMENT.ZCOMMENTTEXT, --4: text of comment
+        ZCLOUDSHAREDCOMMENT.ZCOMMENTERHASHEDPERSONID, --5: hashed ID of person who made comment/like
+        ZCLOUDSHAREDCOMMENT.ZISMYCOMMENT --6: is my (this user's) comment
+        FROM ZCLOUDSHAREDCOMMENT
+        JOIN ZGENERICASSET ON
+        ZGENERICASSET.Z_PK = ZCLOUDSHAREDCOMMENT.ZCOMMENTEDASSET
+        OR
+        ZGENERICASSET.Z_PK = ZCLOUDSHAREDCOMMENT.ZLIKEDASSET
+        WHERE ZGENERICASSET.ZUUID = ?
+    """,
+        ([uuid]),
+    ).fetchall()
+
+    comments = []
+    likes = []
+    for row in results:
+        photo_uuid = row[0]
+        sort_fok = row[2] or 0  # sort_fok is Null/None for likes
+        is_like = bool(row[1])
+        text = row[4]
+        user_info = get_shared_person_info(photosdb, row[5])
+        try:
+            dt = datetime.datetime.fromtimestamp(row[3] + TIME_DELTA)
+        except:
+            dt = datetime.datetime(1970, 1, 1)
+        ismine = bool(row[6])
+        if is_like:
+            # it's a like
+            likes.append(Like(photo_uuid, sort_fok, dt, user_info[2], ismine))
+        elif text:
+            # comment
+            comments.append(
+                Comment(photo_uuid, sort_fok, dt, user_info[2], ismine, text)
+            )
+    if likes:
+        likes.sort(key=lambda x: x.datetime)
+    if comments:
+        comments.sort(key=lambda x: x.sort_fok)
+    return (comments, likes)
+
+
+def main():
+    if len(sys.argv) > 1:
+        # library as first argument
+        photosdb = osxphotos.PhotosDB(dbfile=sys.argv[1])
+    else:
+        # open default library
+        photosdb = osxphotos.PhotosDB()
+
+    # shared albums
+    shared_albums = photosdb.album_info_shared
+    for album in shared_albums:
+        print(f"Processing album {album.title}")
+        # only shared albums can have comments
+        for photo in album.photos:
+            comments, likes = get_comments(photosdb, photo.uuid)
+            if comments or likes:
+                print(f"{photo.uuid}, {photo.original_filename}: ")
+            if likes:
+                print("Likes:")
+                for like in likes:
+                    print(like)
+            if comments:
+                print("Comments:")
+                for comment in comments:
+                    print(comment)
+
+
+if __name__ == "__main__":
+    main()

examples/photos_repl.py

@@ -7,6 +7,7 @@
 # If you run this using python from command line, do so with -i flag:
 # python3 -i examples/photos_repl.py
 
+import os
 import sys
 import time
 
@@ -17,6 +18,23 @@ import osxphotos
 from osxphotos.__main__ import get_photos_db, _list_libraries
 
 
+def show(photo):
+    """ open image with default image viewer
+    
+    Note: This is for debugging only -- it will actually open any filetype which could 
+    be very, very bad.  
+
+    Args:
+        photo: PhotoInfo object or a path to a photo on disk
+    """
+    photopath = photo.path if isinstance(photo, osxphotos.PhotoInfo) else photo
+
+    if not os.path.isfile(photopath):
+        return f"'{photopath}' does not appear to be a valid photo path"
+
+    os.system(f"open '{photopath}'")
+
+
 def main():
     db = None
 
@@ -24,7 +42,7 @@ def main():
     if db:
         print("loading database")
         tic = time.perf_counter()
-        photosdb = osxphotos.PhotosDB(dbfile=db)
+        photosdb = osxphotos.PhotosDB(dbfile=db, verbose=print)
         toc = time.perf_counter()
         print(f"done: took {toc-tic} seconds")
         return photosdb
@@ -40,5 +58,6 @@ if __name__ == "__main__":
     print("getting photos")
     tic = time.perf_counter()
     photos = photosdb.photos(images=True, movies=True)
+    photos.extend(photosdb.photos(images=True, movies=True, intrash=True))
     toc = time.perf_counter()
     print(f"found {len(photos)} photos in {toc-tic} seconds")

osxphotos/__init__.py

@@ -1,16 +1,10 @@
-import logging
-
 from ._version import __version__
 from .photoinfo import PhotoInfo
 from .photosdb import PhotosDB
-from .utils import _set_debug, _debug, _get_logger
+from .photosdb._photosdb_process_comments import CommentInfo, LikeInfo
+from .phototemplate import PhotoTemplate
+from .utils import _debug, _get_logger, _set_debug
 
-# TODO: find edited photos: see https://github.com/orangeturtle739/photos-export/blob/master/extract_photos.py
 # TODO: Add test for imageTimeZoneOffsetSeconds = None
-# TODO: Fix command line so multiple --keyword, etc. are AND (instead of OR as they are in .photos())
-#       Or fix the help text to match behavior
 # TODO: Add test for __str__ and to_json
-# TODO: fix docstrings
 # TODO: Add special albums and magic albums
-# TODO: cleanup os.path and pathlib code (import pathlib and also from pathlib import Path)
-

osxphotos/_constants.py

@@ -3,6 +3,14 @@ Constants used by osxphotos
 """
 
 import os.path
+from datetime import datetime
+
+# Time delta: add this to Photos times to get unix time
+# Apple Epoch is Jan 1, 2001
+TIME_DELTA = (datetime(2001, 1, 1, 0, 0) - datetime(1970, 1, 1, 0, 0)).total_seconds()
+
+# Unicode format to use for comparing strings
+UNICODE_FORMAT = "NFC"
 
 # which Photos library database versions have been tested
 # Photos 2.0 (10.12.6) == 2622
@@ -10,18 +18,48 @@ import os.path
 # Photos 4.0 (10.14.5) == 4016
 # Photos 4.0 (10.14.6) == 4025
 # Photos 5.0 (10.15.0) == 6000
-# TODO: Should this also use compatibleBackToVersion from LiGlobals?
 _TESTED_DB_VERSIONS = ["6000", "4025", "4016", "3301", "2622"]
 
+# database model versions (applies to Photos 5, Photos 6)
+# these come from PLModelVersion key in binary plist in Z_METADATA.Z_PLIST
+# Photos 5 (10.15.1) == 13537
+# Photos 5 (10.15.4, 10.15.5, 10.15.6) == 13703
+# Photos 6 (10.16.0 Beta) == 14104
+_TEST_MODEL_VERSIONS = ["13537", "13703", "14104"]
+
 # only version 3 - 4 have RKVersion.selfPortrait
 _PHOTOS_3_VERSION = "3301"
 
 # versions 5.0 and later have a different database structure
 _PHOTOS_4_VERSION = "4025"  # latest Mojove version on 10.14.6
-_PHOTOS_5_VERSION = "6000"  # seems to be current on 10.15.1 through 10.15.5
+_PHOTOS_5_VERSION = "6000"  # seems to be current on 10.15.1 through 10.15.6
+
+# Ranges for model version by Photos version
+_PHOTOS_5_MODEL_VERSION = [13000, 13999]
+_PHOTOS_6_MODEL_VERSION = [14000, 14999]
+
+# some table names differ between Photos 5 and Photos 6
+_DB_TABLE_NAMES = {
+    5: {
+        "ASSET": "ZGENERICASSET",
+        "KEYWORD_JOIN": "Z_1KEYWORDS.Z_37KEYWORDS",
+        "ALBUM_JOIN": "Z_26ASSETS.Z_34ASSETS",
+        "ALBUM_SORT_ORDER": "Z_26ASSETS.Z_FOK_34ASSETS",
+        "IMPORT_FOK": "ZGENERICASSET.Z_FOK_IMPORTSESSION",
+        "DEPTH_STATE": "ZGENERICASSET.ZDEPTHSTATES",
+    },
+    6: {
+        "ASSET": "ZASSET",
+        "KEYWORD_JOIN": "Z_1KEYWORDS.Z_36KEYWORDS",
+        "ALBUM_JOIN": "Z_26ASSETS.Z_3ASSETS",
+        "ALBUM_SORT_ORDER": "Z_26ASSETS.Z_FOK_3ASSETS",
+        "IMPORT_FOK": "null",
+        "DEPTH_STATE": "ZASSET.ZDEPTHTYPE",
+    },
+}
 
 # which major version operating systems have been tested
-_TESTED_OS_VERSIONS = ["12", "13", "14", "15"]
+_TESTED_OS_VERSIONS = ["12", "13", "14", "15", "16"]
 
 # Photos 5 has persons who are empty string if unidentified face
 _UNKNOWN_PERSON = "_UNKNOWN_"
@@ -47,6 +85,7 @@ _PHOTOS_5_ALBUM_KIND = 2  # normal user album
 _PHOTOS_5_SHARED_ALBUM_KIND = 1505  # shared album
 _PHOTOS_5_FOLDER_KIND = 4000  # user folder
 _PHOTOS_5_ROOT_FOLDER_KIND = 3999  # root folder
+_PHOTOS_5_IMPORT_SESSION_ALBUM_KIND = 1506  # import session
 
 _PHOTOS_4_ALBUM_KIND = 3  # RKAlbum.albumSubclass
 _PHOTOS_4_TOP_LEVEL_ALBUM = "TopLevelAlbums"
@@ -63,3 +102,10 @@ _OSXPHOTOS_NONE_SENTINEL = "OSXPhotosXYZZY42_Sentinel$"
 
 # SearchInfo categories for Photos 5, corresponds to categories in database/search/psi.sqlite
 SEARCH_CATEGORY_LABEL = 2024
+
+# Max filename length on MacOS
+MAX_FILENAME_LEN = 255
+
+# Max directory name length on MacOS
+MAX_DIRNAME_LEN = 255
+

osxphotos/_version.py

@@ -1,3 +1,4 @@
 """ version info """
 
-__version__ = "0.29.16"
+__version__ = "0.36.2"
+

osxphotos/albuminfo.py

@@ -10,7 +10,7 @@ Represents a single Folder in the Photos library and provides access to the fold
 PhotosDB.folders() returns a list of FolderInfo objects
 """
 
-import logging
+from datetime import datetime, timedelta, timezone
 
 from ._constants import (
     _PHOTOS_4_ALBUM_KIND,
@@ -18,11 +18,34 @@ from ._constants import (
     _PHOTOS_4_VERSION,
     _PHOTOS_5_ALBUM_KIND,
     _PHOTOS_5_FOLDER_KIND,
+    TIME_DELTA,
 )
+from .datetime_utils import get_local_tz
 
 
-class AlbumInfo:
+def sort_list_by_keys(values, sort_keys):
+    """ Sorts list values by a second list sort_keys
+        e.g. given ["a","c","b"], [1, 3, 2], returns ["a", "b", "c"]
+
+    Args:
+        values: a list of values to be sorted
+        sort_keys: a list of keys to sort values by
+    
+    Returns:
+        list of values, sorted by sort_keys
+    
+    Raises:
+        ValueError: raised if len(values) != len(sort_keys)
     """
+    if len(values) != len(sort_keys):
+        return ValueError("values and sort_keys must have same length")
+
+    return list(zip(*sorted(zip(sort_keys, values))))[1]
+
+
+class AlbumInfoBaseClass:
+    """
+    Base class for AlbumInfo, ImportInfo
     Info about a specific Album, contains all the details about the album
     including folders, photos, etc.
     """
@@ -31,25 +54,111 @@ class AlbumInfo:
         self._uuid = uuid
         self._db = db
         self._title = self._db._dbalbum_details[uuid]["title"]
-
+        self._creation_date_timestamp = self._db._dbalbum_details[uuid]["creation_date"]
-    @property
+        self._start_date_timestamp = self._db._dbalbum_details[uuid]["start_date"]
-    def title(self):
+        self._end_date_timestamp = self._db._dbalbum_details[uuid]["end_date"]
-        """ return title / name of album """
+        self._local_tz = get_local_tz(
-        return self._title
+            datetime.fromtimestamp(self._creation_date_timestamp + TIME_DELTA)
+        )
 
     @property
     def uuid(self):
         """ return uuid of album """
         return self._uuid
 
+    @property
+    def creation_date(self):
+        """ return creation date of album """
+        try:
+            return self._creation_date
+        except AttributeError:
+            try:
+                self._creation_date = (
+                    datetime.fromtimestamp(
+                        self._creation_date_timestamp + TIME_DELTA
+                    ).astimezone(tz=self._local_tz)
+                    if self._creation_date_timestamp
+                    else datetime(1970, 1, 1, 0, 0, 0).astimezone(
+                        tz=timezone(timedelta(0))
+                    )
+                )
+            except ValueError:
+                self._creation_date = datetime(1970, 1, 1, 0, 0, 0).astimezone(
+                    tz=timezone(timedelta(0))
+                )
+            return self._creation_date
+
+    @property
+    def start_date(self):
+        """ For Albums, return start date (earliest image) of album or None for albums with no images
+            For Import Sessions, return start date of import session (when import began) """
+        try:
+            return self._start_date
+        except AttributeError:
+            try:
+                self._start_date = (
+                    datetime.fromtimestamp(
+                        self._start_date_timestamp + TIME_DELTA
+                    ).astimezone(tz=self._local_tz)
+                    if self._start_date_timestamp
+                    else None
+                )
+            except ValueError:
+                self._start_date = None
+            return self._start_date
+
+    @property
+    def end_date(self):
+        """ For Albums, return end date (most recent image) of album or None for albums with no images
+            For Import Sessions, return end date of import sessions (when import was completed) """
+        try:
+            return self._end_date
+        except AttributeError:
+            try:
+                self._end_date = (
+                    datetime.fromtimestamp(
+                        self._end_date_timestamp + TIME_DELTA
+                    ).astimezone(tz=self._local_tz)
+                    if self._end_date_timestamp
+                    else None
+                )
+            except ValueError:
+                self._end_date = None
+            return self._end_date
+
     @property
     def photos(self):
-        """ return list of photos contained in album """
+        return []
+
+    def __len__(self):
+        """ return number of photos contained in album """
+        return len(self.photos)
+
+
+class AlbumInfo(AlbumInfoBaseClass):
+    """
+    Base class for AlbumInfo, ImportInfo
+    Info about a specific Album, contains all the details about the album
+    including folders, photos, etc.
+    """
+
+    @property
+    def title(self):
+        """ return title / name of album """
+        return self._title
+
+    @property
+    def photos(self):
+        """ return list of photos contained in album sorted in same sort order as Photos """
         try:
             return self._photos
         except AttributeError:
-            uuid = self._db._dbalbums_album[self._uuid]
+            if self.uuid in self._db._dbalbums_album:
-            self._photos = self._db.photos(uuid=uuid)
+                uuid, sort_order = zip(*self._db._dbalbums_album[self.uuid])
+                sorted_uuid = sort_list_by_keys(uuid, sort_order)
+                self._photos = self._db.photos_by_uuid(sorted_uuid)
+            else:
+                self._photos = []
             return self._photos
 
     @property
@@ -100,9 +209,24 @@ class AlbumInfo:
                 )
             return self._parent
 
-    def __len__(self):
+
-        """ return number of photos contained in album """
+class ImportInfo(AlbumInfoBaseClass):
-        return len(self.photos)
+    @property
+    def photos(self):
+        """ return list of photos contained in import session """
+        try:
+            return self._photos
+        except AttributeError:
+            uuid_list, sort_order = zip(
+                *[
+                    (uuid, self._db._dbphotos[uuid]["fok_import_session"])
+                    for uuid in self._db._dbphotos
+                    if self._db._dbphotos[uuid]["import_uuid"] == self.uuid
+                ]
+            )
+            sorted_uuid = sort_list_by_keys(uuid_list, sort_order)
+            self._photos = self._db.photos_by_uuid(sorted_uuid)
+            return self._photos
 
 
 class FolderInfo:

osxphotos/datetime_utils.py

@@ -0,0 +1,62 @@
+""" datetime utilities """
+
+import datetime
+
+
+def get_local_tz(dt):
+    """ return local timezone as datetime.timezone tzinfo for dt
+    
+    Args:
+        dt: datetime.datetime
+    
+    Returns:
+        local timezone for dt as datetime.timezone
+
+    Raises:
+        ValueError if dt is not timezone naive
+    """
+    if not datetime_has_tz(dt):
+        return dt.astimezone().tzinfo
+    else:
+        raise ValueError("dt must be naive datetime.datetime object")
+
+
+def datetime_remove_tz(dt):
+    """ remove timezone from a datetime.datetime object
+        dt: datetime.datetime object with tzinfo
+        returns: dt without any timezone info (naive datetime object) """
+
+    if type(dt) != datetime.datetime:
+        raise TypeError(f"dt must be type datetime.datetime, not {type(dt)}")
+
+    return dt.replace(tzinfo=None)
+
+
+def datetime_has_tz(dt):
+    """ return True if datetime dt has tzinfo else False
+        dt: datetime.datetime
+        returns True if dt is timezone aware, else False """
+
+    if type(dt) != datetime.datetime:
+        raise TypeError(f"dt must be type datetime.datetime, not {type(dt)}")
+
+    return dt.tzinfo is not None and dt.tzinfo.utcoffset(dt) is not None
+
+
+def datetime_naive_to_local(dt):
+    """ convert naive (timezone unaware) datetime.datetime
+        to aware timezone in local timezone
+        dt: datetime.datetime without timezone
+        returns: datetime.datetime with local timezone """
+
+    if type(dt) != datetime.datetime:
+        raise TypeError(f"dt must be type datetime.datetime, not {type(dt)}")
+
+    if dt.tzinfo is not None and dt.tzinfo.utcoffset(dt) is not None:
+        # has timezone info
+        raise ValueError(
+            "dt must be naive/timezone unaware: "
+            f"{dt} has tzinfo {dt.tzinfo} and offset {dt.tizinfo.utcoffset(dt)}"
+        )
+
+    return dt.replace(tzinfo=get_local_tz(dt))

osxphotos/exiftool.py

@@ -8,6 +8,7 @@
 import json
 import logging
 import os
+import shutil
 import subprocess
 import sys
 from functools import lru_cache  # pylint: disable=syntax-error
@@ -22,8 +23,7 @@ EXIFTOOL_STAYOPEN_EOF_LEN = len(EXIFTOOL_STAYOPEN_EOF)
 @lru_cache(maxsize=1)
 def get_exiftool_path():
     """ return path of exiftool, cache result """
-    result = subprocess.run(["which", "exiftool"], stdout=subprocess.PIPE)
+    exiftool_path = shutil.which('exiftool')
-    exiftool_path = result.stdout.decode("utf-8")
     if _debug():
         logging.debug("exiftool path = %s" % (exiftool_path))
     if exiftool_path:
@@ -98,6 +98,7 @@ class _ExifToolProc:
                 "-",  # read from stdin
                 "-common_args",  # specifies args common to all commands subsequently run
                 "-n",  # no print conversion (e.g. print tag values in machine readable format)
+                "-P",  # Preserve file modification date/time (possible interfere w/ --touch-file)
                 "-G",  # print group name for each tag
             ],
             stdin=subprocess.PIPE,
@@ -227,7 +228,7 @@ class ExifTool:
         ver = self.run_commands("-ver", no_file=True)
         return ver.decode("utf-8")
 
-    def as_dict(self):
+    def asdict(self):
         """ return dictionary of all EXIF tags and values from exiftool 
             returns empty dict if no tags
         """
@@ -244,7 +245,7 @@ class ExifTool:
 
     def _read_exif(self):
         """ read exif data from file """
-        data = self.as_dict()
+        data = self.asdict()
         self.data = {k: v for k, v in data.items()}
 
     def __str__(self):

osxphotos/export_db.py

@@ -14,7 +14,7 @@ from sqlite3 import Error
 
 from ._version import __version__
 
-OSXPHOTOS_EXPORTDB_VERSION = "1.0"
+OSXPHOTOS_EXPORTDB_VERSION = "2.0"
 
 
 class ExportDB_ABC(ABC):
@@ -36,6 +36,22 @@ class ExportDB_ABC(ABC):
     def get_stat_orig_for_file(self, filename):
         pass
 
+    @abstractmethod
+    def set_stat_edited_for_file(self, filename, stats):
+        pass
+
+    @abstractmethod
+    def get_stat_edited_for_file(self, filename):
+        pass
+
+    @abstractmethod
+    def set_stat_converted_for_file(self, filename, stats):
+        pass
+
+    @abstractmethod
+    def get_stat_converted_for_file(self, filename):
+        pass
+
     @abstractmethod
     def set_stat_exif_for_file(self, filename, stats):
         pass
@@ -61,13 +77,28 @@ class ExportDB_ABC(ABC):
         pass
 
     @abstractmethod
-    def set_data(self, filename, uuid, orig_stat, exif_stat, info_json, exif_json):
+    def set_data(
+        self,
+        filename,
+        uuid,
+        orig_stat,
+        exif_stat,
+        converted_stat,
+        edited_stat,
+        info_json,
+        exif_json,
+    ):
         pass
 
 
 class ExportDBNoOp(ExportDB_ABC):
     """ An ExportDB with NoOp methods """
 
+    def __init__(self):
+        self.was_created = True
+        self.was_upgraded = False
+        self.version = OSXPHOTOS_EXPORTDB_VERSION
+
     def get_uuid_for_file(self, filename):
         pass
 
@@ -80,6 +111,18 @@ class ExportDBNoOp(ExportDB_ABC):
     def get_stat_orig_for_file(self, filename):
         pass
 
+    def set_stat_edited_for_file(self, filename, stats):
+        pass
+
+    def get_stat_edited_for_file(self, filename):
+        pass
+
+    def set_stat_converted_for_file(self, filename, stats):
+        pass
+
+    def get_stat_converted_for_file(self, filename):
+        pass
+
     def set_stat_exif_for_file(self, filename, stats):
         pass
 
@@ -98,7 +141,17 @@ class ExportDBNoOp(ExportDB_ABC):
     def set_exifdata_for_file(self, uuid, exifdata):
         pass
 
-    def set_data(self, filename, uuid, orig_stat, exif_stat, info_json, exif_json):
+    def set_data(
+        self,
+        filename,
+        uuid,
+        orig_stat,
+        exif_stat,
+        converted_stat,
+        edited_stat,
+        info_json,
+        exif_json,
+    ):
         pass
 
 
@@ -122,7 +175,6 @@ class ExportDB(ExportDB_ABC):
             returns None if filename not found in database
         """
         filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
-        logging.debug(f"get_uuid: {filename}")
         conn = self._conn
         try:
             c = conn.cursor()
@@ -135,14 +187,12 @@ class ExportDB(ExportDB_ABC):
             logging.warning(e)
             uuid = None
 
-        logging.debug(f"get_uuid: {uuid}")
         return uuid
 
     def set_uuid_for_file(self, filename, uuid):
         """ set UUID of filename to uuid in the database """
         filename = str(pathlib.Path(filename).relative_to(self._path))
         filename_normalized = filename.lower()
-        logging.debug(f"set_uuid: {filename} {uuid}")
         conn = self._conn
         try:
             c = conn.cursor()
@@ -162,7 +212,6 @@ class ExportDB(ExportDB_ABC):
         if len(stats) != 3:
             raise ValueError(f"expected 3 elements for stat, got {len(stats)}")
 
-        logging.debug(f"set_stat_orig_for_file: {filename} {stats}")
         conn = self._conn
         try:
             c = conn.cursor()
@@ -189,14 +238,30 @@ class ExportDB(ExportDB_ABC):
                 (filename,),
             )
             results = c.fetchone()
-            stats = results[0:3] if results else None
+            if results:
+                stats = results[0:3]
+                mtime = int(stats[2]) if stats[2] is not None else None
+                stats = (stats[0], stats[1], mtime)
+            else:
+                stats = (None, None, None)
         except Error as e:
             logging.warning(e)
             stats = (None, None, None)
 
-        logging.debug(f"get_stat_orig_for_file: {stats}")
         return stats
 
+    def set_stat_edited_for_file(self, filename, stats):
+        """ set stat info for edited version of image (in Photos' library)
+            filename: filename to set the stat info for
+            stat: a tuple of length 3: mode, size, mtime """
+        return self._set_stat_for_file("edited", filename, stats)
+
+    def get_stat_edited_for_file(self, filename):
+        """ get stat info for edited version of image (in Photos' library)
+            filename: filename to set the stat info for
+            stat: a tuple of length 3: mode, size, mtime """
+        return self._get_stat_for_file("edited", filename)
+
     def set_stat_exif_for_file(self, filename, stats):
         """ set stat info for filename (after exiftool has updated it)
             filename: filename to set the stat info for
@@ -205,7 +270,6 @@ class ExportDB(ExportDB_ABC):
         if len(stats) != 3:
             raise ValueError(f"expected 3 elements for stat, got {len(stats)}")
 
-        logging.debug(f"set_stat_exif_for_file: {filename} {stats}")
         conn = self._conn
         try:
             c = conn.cursor()
@@ -232,14 +296,30 @@ class ExportDB(ExportDB_ABC):
                 (filename,),
             )
             results = c.fetchone()
-            stats = results[0:3] if results else None
+            if results:
+                stats = results[0:3]
+                mtime = int(stats[2]) if stats[2] is not None else None
+                stats = (stats[0], stats[1], mtime)
+            else:
+                stats = (None, None, None)
         except Error as e:
             logging.warning(e)
             stats = (None, None, None)
 
-        logging.debug(f"get_stat_exif_for_file: {stats}")
         return stats
 
+    def set_stat_converted_for_file(self, filename, stats):
+        """ set stat info for filename (after image converted to jpeg)
+            filename: filename to set the stat info for
+            stat: a tuple of length 3: mode, size, mtime """
+        return self._set_stat_for_file("converted", filename, stats)
+
+    def get_stat_converted_for_file(self, filename):
+        """ get stat info for filename (after jpeg conversion)
+            returns: tuple of (mode, size, mtime) 
+        """
+        return self._get_stat_for_file("converted", filename)
+
     def get_info_for_uuid(self, uuid):
         """ returns the info JSON struct for a UUID """
         conn = self._conn
@@ -252,7 +332,6 @@ class ExportDB(ExportDB_ABC):
             logging.warning(e)
             info = None
 
-        logging.debug(f"get_info: {uuid}, {info}")
         return info
 
     def set_info_for_uuid(self, uuid, info):
@@ -268,8 +347,6 @@ class ExportDB(ExportDB_ABC):
         except Error as e:
             logging.warning(e)
 
-        logging.debug(f"set_info: {uuid}, {info}")
-
     def get_exifdata_for_file(self, filename):
         """ returns the exifdata JSON struct for a file """
         filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
@@ -286,7 +363,6 @@ class ExportDB(ExportDB_ABC):
             logging.warning(e)
             exifdata = None
 
-        logging.debug(f"get_exifdata: {filename}, {exifdata}")
         return exifdata
 
     def set_exifdata_for_file(self, filename, exifdata):
@@ -303,9 +379,17 @@ class ExportDB(ExportDB_ABC):
         except Error as e:
             logging.warning(e)
 
-        logging.debug(f"set_exifdata: {filename}, {exifdata}")
+    def set_data(
-
+        self,
-    def set_data(self, filename, uuid, orig_stat, exif_stat, info_json, exif_json):
+        filename,
+        uuid,
+        orig_stat,
+        exif_stat,
+        converted_stat,
+        edited_stat,
+        info_json,
+        exif_json,
+    ):
         """ sets all the data for file and uuid at once 
         """
         filename = str(pathlib.Path(filename).relative_to(self._path))
@@ -329,6 +413,14 @@ class ExportDB(ExportDB_ABC):
                 + "WHERE filepath_normalized = ?;",
                 (*exif_stat, filename_normalized),
             )
+            c.execute(
+                "INSERT OR REPLACE INTO converted(filepath_normalized, mode, size, mtime) VALUES (?, ?, ?, ?);",
+                (filename_normalized, *converted_stat),
+            )
+            c.execute(
+                "INSERT OR REPLACE INTO edited(filepath_normalized, mode, size, mtime) VALUES (?, ?, ?, ?);",
+                (filename_normalized, *edited_stat),
+            )
             c.execute(
                 "INSERT OR REPLACE INTO info(uuid, json_info) VALUES (?, ?);",
                 (uuid, info_json),
@@ -348,6 +440,37 @@ class ExportDB(ExportDB_ABC):
         except Error as e:
             logging.warning(e)
 
+    def _set_stat_for_file(self, table, filename, stats):
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        if len(stats) != 3:
+            raise ValueError(f"expected 3 elements for stat, got {len(stats)}")
+
+        conn = self._conn
+        c = conn.cursor()
+        c.execute(
+            f"INSERT OR REPLACE INTO {table}(filepath_normalized, mode, size, mtime) VALUES (?, ?, ?, ?);",
+            (filename, *stats),
+        )
+        conn.commit()
+
+    def _get_stat_for_file(self, table, filename):
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        conn = self._conn
+        c = conn.cursor()
+        c.execute(
+            f"SELECT mode, size, mtime FROM {table} WHERE filepath_normalized = ?",
+            (filename,),
+        )
+        results = c.fetchone()
+        if results:
+            stats = results[0:3]
+            mtime = int(stats[2]) if stats[2] is not None else None
+            stats = (stats[0], stats[1], mtime)
+        else:
+            stats = (None, None, None)
+
+        return stats
+
     def _open_export_db(self, dbfile):
         """ open export database and return a db connection
             if dbfile does not exist, will create and initialize the database 
@@ -355,15 +478,24 @@ class ExportDB(ExportDB_ABC):
         """
 
         if not os.path.isfile(dbfile):
-            logging.debug(f"dbfile {dbfile} doesn't exist, creating it")
             conn = self._get_db_connection(dbfile)
             if conn:
                 self._create_db_tables(conn)
+                self.was_created = True
+                self.was_upgraded = ()
+                self.version = OSXPHOTOS_EXPORTDB_VERSION
             else:
                 raise Exception("Error getting connection to database {dbfile}")
         else:
-            logging.debug(f"dbfile {dbfile} exists, opening it")
             conn = self._get_db_connection(dbfile)
+            self.was_created = False
+            version_info = self._get_database_version(conn)
+            if version_info[1] < OSXPHOTOS_EXPORTDB_VERSION:
+                self._create_db_tables(conn)
+                self.was_upgraded = (version_info[1], OSXPHOTOS_EXPORTDB_VERSION)
+            else:
+                self.was_upgraded = ()
+            self.version = OSXPHOTOS_EXPORTDB_VERSION
 
         return conn
 
@@ -377,6 +509,13 @@ class ExportDB(ExportDB_ABC):
 
         return conn
 
+    def _get_database_version(self, conn):
+        """ return tuple of (osxphotos, exportdb) versions for database connection conn """
+        version_info = conn.execute(
+            "SELECT osxphotos, exportdb, max(id) FROM version"
+        ).fetchone()
+        return (version_info[0], version_info[1])
+
     def _create_db_tables(self, conn):
         """ create (if not already created) the necessary db tables for the export database
             conn: sqlite3 db connection 
@@ -417,9 +556,25 @@ class ExportDB(ExportDB_ABC):
                              filepath_normalized TEXT NOT NULL,
                              json_exifdata JSON 
                              ); """,
-            "sql_files_idx": """ CREATE UNIQUE INDEX idx_files_filepath_normalized on files (filepath_normalized); """,
+            "sql_edited_table": """ CREATE TABLE IF NOT EXISTS edited (
-            "sql_info_idx": """ CREATE UNIQUE INDEX idx_info_uuid on info (uuid); """,
+                              id INTEGER PRIMARY KEY,
-            "sql_exifdata_idx": """ CREATE UNIQUE INDEX idx_exifdata_filename on exifdata (filepath_normalized); """,
+                              filepath_normalized TEXT NOT NULL,
+                              mode INTEGER,
+                              size INTEGER,
+                              mtime REAL
+                              ); """,
+            "sql_converted_table": """ CREATE TABLE IF NOT EXISTS converted (
+                              id INTEGER PRIMARY KEY,
+                              filepath_normalized TEXT NOT NULL,
+                              mode INTEGER,
+                              size INTEGER,
+                              mtime REAL
+                              ); """,
+            "sql_files_idx": """ CREATE UNIQUE INDEX IF NOT EXISTS idx_files_filepath_normalized on files (filepath_normalized); """,
+            "sql_info_idx": """ CREATE UNIQUE INDEX IF NOT EXISTS idx_info_uuid on info (uuid); """,
+            "sql_exifdata_idx": """ CREATE UNIQUE INDEX IF NOT EXISTS idx_exifdata_filename on exifdata (filepath_normalized); """,
+            "sql_edited_idx": """ CREATE UNIQUE INDEX IF NOT EXISTS idx_edited_filename on edited (filepath_normalized);""",
+            "sql_converted_idx": """ CREATE UNIQUE INDEX IF NOT EXISTS idx_converted_filename on converted (filepath_normalized);""",
         }
         try:
             c = conn.cursor()
@@ -435,11 +590,10 @@ class ExportDB(ExportDB_ABC):
 
     def __del__(self):
         """ ensure the database connection is closed """
-        if self._conn:
+        try:
-            try:
+            self._conn.close()
-                self._conn.close()
+        except:
-            except Error as e:
+            pass
-                logging.warning(e)
 
     def _insert_run_info(self):
         dt = datetime.datetime.utcnow().isoformat()
@@ -478,18 +632,18 @@ class ExportDBInMemory(ExportDB):
 
     def _open_export_db(self, dbfile):
         """ open export database and return a db connection
-            if dbfile does not exist, will create and initialize the database 
             returns: connection to the database 
         """
         if not os.path.isfile(dbfile):
-            logging.debug(f"dbfile {dbfile} doesn't exist, creating in memory version")
             conn = self._get_db_connection()
             if conn:
                 self._create_db_tables(conn)
+                self.was_created = True
+                self.was_upgraded = ()
+                self.version = OSXPHOTOS_EXPORTDB_VERSION
             else:
                 raise Exception("Error getting connection to in-memory database")
         else:
-            logging.debug(f"dbfile {dbfile} exists, opening it and copying to memory")
             try:
                 conn = sqlite3.connect(dbfile)
             except Error as e:
@@ -506,6 +660,14 @@ class ExportDBInMemory(ExportDB):
             conn = sqlite3.connect(":memory:")
             conn.cursor().executescript(tempfile.read())
             conn.commit()
+            self.was_created = False
+            _, exportdb_ver = self._get_database_version(conn)
+            if exportdb_ver < OSXPHOTOS_EXPORTDB_VERSION:
+                self._create_db_tables(conn)
+                self.was_upgraded = (exportdb_ver, OSXPHOTOS_EXPORTDB_VERSION)
+            else:
+                self.was_upgraded = ()
+            self.version = OSXPHOTOS_EXPORTDB_VERSION
 
         return conn
 

osxphotos/fileutil.py

@@ -8,6 +8,7 @@ import subprocess
 import sys
 from abc import ABC, abstractmethod
 
+from .imageconverter import ImageConverter
 
 class FileUtilABC(ABC):
     """ Abstract base class for FileUtil """
@@ -29,7 +30,17 @@ class FileUtilABC(ABC):
 
     @classmethod
     @abstractmethod
-    def cmp_sig(cls, file1, file2):
+    def utime(cls, path, times):
+        pass
+
+    @classmethod
+    @abstractmethod
+    def cmp(cls, file1, file2, mtime1=None):
+        pass
+
+    @classmethod
+    @abstractmethod
+    def cmp_file_sig(cls, file1, file2):
         pass
 
     @classmethod
@@ -37,6 +48,11 @@ class FileUtilABC(ABC):
     def file_sig(cls, file1):
         pass
 
+    @classmethod
+    @abstractmethod
+    def convert_to_jpeg(cls, src_file, dest_file, compression_quality=1.0):
+        pass
+
 
 class FileUtilMacOS(FileUtilABC):
     """ Various file utilities """
@@ -104,11 +120,37 @@ class FileUtilMacOS(FileUtilABC):
             os.unlink(filepath)
 
     @classmethod
-    def cmp_sig(cls, f1, s2):
+    def utime(cls, path, times):
+        """ Set the access and modified time of path. """
+        os.utime(path, times)
+
+    @classmethod
+    def cmp(cls, f1, f2, mtime1=None):
+        """Does shallow compare (file signatures) of f1 to file f2.
+        Arguments:
+        f1 --  File name
+        f2 -- File name
+        mtime1 -- optional, pass alternate file modification timestamp for f1; will be converted to int
+
+        Return value:
+        True if the file signatures as returned by stat are the same, False otherwise. 
+        Does not do a byte-by-byte comparison.
+        """
+
+        s1 = cls._sig(os.stat(f1))
+        if mtime1 is not None:
+            s1 = (s1[0], s1[1], int(mtime1))
+        s2 = cls._sig(os.stat(f2))
+        if s1[0] != stat.S_IFREG or s2[0] != stat.S_IFREG:
+            return False
+        return s1 == s2
+
+    @classmethod
+    def cmp_file_sig(cls, f1, s2):
         """Compare file f1 to signature s2.
         Arguments:
         f1 --  File name
-        s2 -- stats as returned by sig
+        s2 -- stats as returned by _sig
 
         Return value:
         True if the files are the same, False otherwise.
@@ -128,10 +170,29 @@ class FileUtilMacOS(FileUtilABC):
         """ return os.stat signature for file f1 """
         return cls._sig(os.stat(f1))
     
+    @classmethod
+    def convert_to_jpeg(cls, src_file, dest_file, compression_quality=1.0):
+        """ converts image file src_file to jpeg format as dest_file
+
+            Args:
+                src_file: image file to convert
+                dest_file: destination path to write converted file to
+                compression quality: JPEG compression quality in range 0.0 <= compression_quality <= 1.0; default 1.0 (best quality)
+                
+            Returns:
+                True if success, otherwise False 
+        """
+        converter = ImageConverter()
+        return converter.write_jpeg(src_file, dest_file, compression_quality=compression_quality)
+
     @staticmethod
     def _sig(st):
-        return (stat.S_IFMT(st.st_mode), st.st_size, st.st_mtime)
+        """ return tuple of (mode, size, mtime) of file based on os.stat
-
+            Args:
+                st: os.stat signature
+        """
+        # use int(st.st_mtime) because ditto does not copy fractional portion of mtime
+        return (stat.S_IFMT(st.st_mode), st.st_size, int(st.st_mtime))
 
 class FileUtil(FileUtilMacOS):
     """ Various file utilities """
@@ -141,8 +202,8 @@ class FileUtil(FileUtilMacOS):
 
 class FileUtilNoOp(FileUtil):
     """ No-Op implementation of FileUtil for testing / dry-run mode
-        all methods with exception of cmp_sig and file_cmp are no-op
+        all methods with exception of cmp, cmp_file_sig and file_cmp are no-op
-        cmp_sig functions as FileUtil.cmp_sig does
+        cmp and cmp_file_sig functions as FileUtil methods do
         file_cmp returns mock data
     """
 
@@ -172,7 +233,15 @@ class FileUtilNoOp(FileUtil):
     def unlink(cls, dest):
         cls.verbose(f"unlink: {dest}")
 
+    @classmethod
+    def utime(cls, path, times):
+        cls.verbose(f"utime: {path}, {times}")
+
     @classmethod
     def file_sig(cls, file1):
         cls.verbose(f"file_sig: {file1}")
         return (42, 42, 42)
+
+    @classmethod
+    def convert_to_jpeg(cls, src_file, dest_file, compression_quality=1.0):
+        cls.verbose(f"convert_to_jpeg: {src_file}, {dest_file}, {compression_quality}")

osxphotos/imageconverter.py

@@ -0,0 +1,112 @@
+""" ImageConverter class
+    Convert an image to JPEG using CoreImage -- 
+    for example, RAW to JPEG.  Only works if Mac equipped with GPU. """
+
+# reference: https://stackoverflow.com/questions/59330149/coreimage-ciimage-write-jpg-is-shifting-colors-macos/59334308#59334308
+
+import logging
+import pathlib
+
+import Metal
+import Quartz
+from Cocoa import NSURL
+from Foundation import NSDictionary
+
+# needed to capture system-level stderr
+from wurlitzer import pipes
+
+
+class ImageConverter:
+    """ Convert images to jpeg.  This class is a singleton
+        which will re-use the Core Image CIContext to avoid
+        creating a new context for every conversion. """
+
+    def __new__(cls, *args, **kwargs):
+        """ create new object or return instance of already created singleton """
+        if not hasattr(cls, "instance") or not cls.instance:
+            cls.instance = super().__new__(cls)
+
+        return cls.instance
+
+    def __init__(self):
+        """ return existing singleton or create a new one """
+
+        if hasattr(self, "context"):
+            return
+
+        """ initialize CIContext """
+        context_options = NSDictionary.dictionaryWithDictionary_(
+            {
+                "workingColorSpace": Quartz.CoreGraphics.kCGColorSpaceExtendedSRGB,
+                "workingFormat": Quartz.kCIFormatRGBAh,
+            }
+        )
+        mtldevice = Metal.MTLCreateSystemDefaultDevice()
+        self.context = Quartz.CIContext.contextWithMTLDevice_options_(
+            mtldevice, context_options
+        )
+
+    def write_jpeg(self, input_path, output_path, compression_quality=1.0):
+        """ convert image to jpeg and write image to output_path
+
+        Args:
+            input_path: path to input image (e.g. '/path/to/import/file.CR2') as str or pathlib.Path
+            output_path: path to exported jpeg (e.g. '/path/to/export/file.jpeg') as str or pathlib.Path
+            compression_quality: JPEG compression quality, float in range 0.0 to 1.0; default is 1.0 (best quality)
+
+        Return:
+            True if conversion successful, else False
+
+        Raises:
+            ValueError if compression quality not in range 0.0 to 1.0
+            FileNotFoundError if input_path doesn't exist
+        """
+
+        # accept input_path or output_path as pathlib.Path
+        if not isinstance(input_path, str):
+            input_path = str(input_path)
+
+        if not isinstance(output_path, str):
+            output_path = str(output_path)
+
+        if not pathlib.Path(input_path).is_file():
+            raise FileNotFoundError(f"could not find {input_path}")
+
+        if not (0.0 <= compression_quality <= 1.0):
+            raise ValueError(
+                "illegal value for compression_quality: {compression_quality}"
+            )
+
+        input_url = NSURL.fileURLWithPath_(input_path)
+        output_url = NSURL.fileURLWithPath_(output_path)
+
+        with pipes() as (out, err):
+            # capture stdout and stderr from system calls
+            # otherwise, Quartz.CIImage.imageWithContentsOfURL_
+            # prints to stderr something like:
+            # 2020-09-20 20:55:25.538 python[73042:5650492] Creating client/daemon connection: B8FE995E-3F27-47F4-9FA8-559C615FD774
+            # 2020-09-20 20:55:25.652 python[73042:5650492] Got the query meta data reply for: com.apple.MobileAsset.RawCamera.Camera, response: 0
+            input_image = Quartz.CIImage.imageWithContentsOfURL_(input_url)
+
+        if input_image is None:
+            logging.debug(f"Could not create CIImage for {input_path}")
+            return False
+
+        output_colorspace = input_image.colorSpace() or Quartz.CGColorSpaceCreateWithName(
+            Quartz.CoreGraphics.kCGColorSpaceSRGB
+        )
+
+        output_options = NSDictionary.dictionaryWithDictionary_(
+            {"kCGImageDestinationLossyCompressionQuality": compression_quality}
+        )
+        _, error = self.context.writeJPEGRepresentationOfImage_toURL_colorSpace_options_error_(
+            input_image, output_url, output_colorspace, output_options, None
+        )
+        if not error:
+            return True
+        else:
+            logging.debug(
+                "Error converting file {input_path} to jpeg at {output_path}: {error}"
+            )
+            return False
+

osxphotos/path_utils.py

@@ -0,0 +1,78 @@
+""" utility functions for validating/sanitizing path components """
+
+from ._constants import MAX_DIRNAME_LEN, MAX_FILENAME_LEN
+import pathvalidate
+
+
+def sanitize_filepath(filepath):
+    """ sanitize a filepath """
+    return pathvalidate.sanitize_filepath(filepath, platform="macos")
+
+
+def is_valid_filepath(filepath):
+    """ returns True if a filepath is valid otherwise False """
+    return pathvalidate.is_valid_filepath(filepath, platform="macos")
+
+
+def sanitize_filename(filename, replacement=":"):
+    """ replace any illegal characters in a filename and truncate filename if needed
+
+    Args:
+        filename: str, filename to sanitze
+        replacement: str, value to replace any illegal characters with; default = ":"
+    
+    Returns:
+        filename with any illegal characters replaced by replacement and truncated if necessary
+    """
+
+    if filename:
+        filename = filename.replace("/", replacement)
+        if len(filename) > MAX_FILENAME_LEN:
+            parts = filename.split(".")
+            drop = len(filename) - MAX_FILENAME_LEN
+            if len(parts) > 1:
+                # has an extension
+                ext = parts.pop(-1)
+                stem = ".".join(parts)
+                if drop > len(stem):
+                    ext = ext[:-drop]
+                else:
+                    stem = stem[:-drop]
+                filename = f"{stem}.{ext}"
+            else:
+                filename = filename[:-drop]
+    return filename
+
+
+def sanitize_dirname(dirname, replacement=":"):
+    """ replace any illegal characters in a directory name and truncate directory name if needed
+
+    Args:
+        dirname: str, directory name to sanitze
+        replacement: str, value to replace any illegal characters with; default = ":"
+    
+    Returns:
+        dirname with any illegal characters replaced by replacement and truncated if necessary
+    """
+    if dirname:
+        dirname = sanitize_pathpart(dirname, replacement=replacement)
+    return dirname
+
+
+def sanitize_pathpart(pathpart, replacement=":"):
+    """ replace any illegal characters in a path part (either directory or filename without extension) and truncate name if needed
+
+    Args:
+        pathpart: str, path part to sanitze
+        replacement: str, value to replace any illegal characters with; default = ":"
+    
+    Returns:
+        pathpart with any illegal characters replaced by replacement and truncated if necessary
+    """
+    if pathpart:
+        pathpart = pathpart.replace("/", replacement)
+        if len(pathpart) > MAX_DIRNAME_LEN:
+            drop = len(pathpart) - MAX_DIRNAME_LEN
+            pathpart = pathpart[:-drop]
+    return pathpart
+

osxphotos/personinfo.py

@@ -0,0 +1,411 @@
+""" PhotoInfo and FaceInfo classes to expose info about persons and faces in the Photos library """
+
+import json
+import logging
+import math
+
+
+class PersonInfo:
+    """ Info about a person in the Photos library
+    """
+
+    def __init__(self, db=None, pk=None):
+        """ Creates a new PersonInfo instance
+
+        Arguments:
+            db: instance of PhotosDB object
+            pk: primary key value of person to initialize PersonInfo with    
+        
+        Returns:
+            PersonInfo instance
+        """
+        self._db = db
+        self._pk = pk
+
+        person = self._db._dbpersons_pk[pk]
+        self.uuid = person["uuid"]
+        self.name = person["fullname"]
+        self.display_name = person["displayname"]
+        self.keyface = person["keyface"]
+        self.facecount = person["facecount"]
+
+    @property
+    def keyphoto(self):
+        try:
+            return self._keyphoto
+        except AttributeError:
+            person = self._db._dbpersons_pk[self._pk]
+            if person["photo_uuid"]:
+                try:
+                    key_photo = self._db.get_photo(person["photo_uuid"])
+                except IndexError:
+                    key_photo = None
+            else:
+                key_photo = None
+            self._keyphoto = key_photo
+            return self._keyphoto
+
+    @property
+    def photos(self):
+        """ Returns list of PhotoInfo objects associated with this person """
+        return self._db.photos_by_uuid(self._db._dbfaces_pk[self._pk])
+
+    @property
+    def face_info(self):
+        """ Returns a list of FaceInfo objects associated with this person sorted by quality score
+            Highest quality face is result[0] and lowest quality face is result[n] 
+        """
+        try:
+            faces = self._db._db_faceinfo_person[self._pk]
+            return sorted(
+                [FaceInfo(db=self._db, pk=face) for face in faces],
+                key=lambda face: face.quality,
+                reverse=True,
+            )
+        except KeyError:
+            # no faces
+            return []
+
+    def asdict(self):
+        """ Returns dictionary representation of class instance """
+        keyphoto = self.keyphoto.uuid if self.keyphoto is not None else None
+        return {
+            "uuid": self.uuid,
+            "name": self.name,
+            "displayname": self.display_name,
+            "keyface": self.keyface,
+            "facecount": self.facecount,
+            "keyphoto": keyphoto,
+        }
+
+    def json(self):
+        """ Returns JSON representation of class instance """
+        return json.dumps(self.asdict())
+
+    def __str__(self):
+        return f"PersonInfo(name={self.name}, display_name={self.display_name}, uuid={self.uuid}, facecount={self.facecount})"
+
+    def __eq__(self, other):
+        if not isinstance(other, type(self)):
+            return False
+
+        return all(
+            getattr(self, field) == getattr(other, field) for field in ["_db", "_pk"]
+        )
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+
+class FaceInfo:
+    """ Info about a face in the Photos library
+    """
+
+    def __init__(self, db=None, pk=None):
+        """ Creates a new FaceInfo instance
+
+        Arguments:
+            db: instance of PhotosDB object
+            pk: primary key value of face to init the object with    
+        
+        Returns:
+            FaceInfo instance
+        """
+        self._db = db
+        self._pk = pk
+
+        face = self._db._db_faceinfo_pk[pk]
+        self._info = face
+        self.uuid = face["uuid"]
+        self.name = face["fullname"]
+        self.asset_uuid = face["asset_uuid"]
+        self._person_pk = face["person"]
+        self.center_x = face["centerx"]
+        self.center_y = face["centery"]
+        self.mouth_x = face["mouthx"]
+        self.mouth_y = face["mouthy"]
+        self.left_eye_x = face["lefteyex"]
+        self.left_eye_y = face["lefteyey"]
+        self.right_eye_x = face["righteyex"]
+        self.right_eye_y = face["righteyey"]
+        self.size = face["size"]
+        self.quality = face["quality"]
+        self.source_width = face["sourcewidth"]
+        self.source_height = face["sourceheight"]
+        self.has_smile = face["has_smile"]
+        self.left_eye_closed = face["left_eye_closed"]
+        self.right_eye_closed = face["right_eye_closed"]
+        self.manual = face["manual"]
+        self.face_type = face["facetype"]
+        self.age_type = face["agetype"]
+        self.bald_type = face["baldtype"]
+        self.eye_makeup_type = face["eyemakeuptype"]
+        self.eye_state = face["eyestate"]
+        self.facial_hair_type = face["facialhairtype"]
+        self.gender_type = face["gendertype"]
+        self.glasses_type = face["glassestype"]
+        self.hair_color_type = face["haircolortype"]
+        self.intrash = face["intrash"]
+        self.lip_makeup_type = face["lipmakeuptype"]
+        self.smile_type = face["smiletype"]
+
+    @property
+    def center(self):
+        """ Coordinates, in PIL format, for center of face 
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point((self.center_x, self.center_y))
+
+    @property
+    def size_pixels(self):
+        """ Size of face in pixels (centered around center_x, center_y)
+
+        Returns:
+            size, in int pixels, of a circle drawn around the center of the face
+        """
+        photo = self.photo
+        size_reference = photo.width if photo.width > photo.height else photo.height
+        return self.size * size_reference
+
+    @property
+    def mouth(self):
+        """ Coordinates, in PIL format, for mouth position
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point_with_rotation((self.mouth_x, self.mouth_y))
+
+    @property
+    def left_eye(self):
+        """ Coordinates, in PIL format, for left eye position
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point_with_rotation((self.left_eye_x, self.left_eye_y))
+
+    @property
+    def right_eye(self):
+        """ Coordinates, in PIL format, for right eye position
+
+        Returns:
+            tuple of coordinates in form (x, y)
+        """
+        return self._make_point_with_rotation((self.right_eye_x, self.right_eye_y))
+
+    @property
+    def person_info(self):
+        """ PersonInfo instance for person associated with this face """
+        try:
+            return self._person
+        except AttributeError:
+            self._person = PersonInfo(db=self._db, pk=self._person_pk)
+            return self._person
+
+    @property
+    def photo(self):
+        """ PhotoInfo instance associated with this face """
+        try:
+            return self._photo
+        except AttributeError:
+            self._photo = self._db.get_photo(self.asset_uuid)
+            if self._photo is None:
+                logging.warning(f"Could not get photo for uuid: {self.asset_uuid}")
+            return self._photo
+
+    def face_rect(self):
+        """ Get face rectangle coordinates for current version of the associated image
+            If image has been edited, rectangle applies to edited version, otherwise original version
+            Coordinates in format and reference frame used by PIL
+
+        Returns:
+            list [(x0, x1), (y0, y1)] of coordinates in reference frame used by PIL
+        """
+        photo = self.photo
+        size_reference = photo.width if photo.width > photo.height else photo.height
+        radius = (self.size / 2) * size_reference
+        x, y = self._make_point((self.center_x, self.center_y))
+        x0, y0 = x - radius, y - radius
+        x1, y1 = x + radius, y + radius
+        return [(x0, y0), (x1, y1)]
+
+    def roll_pitch_yaw(self):
+        """ Roll, pitch, yaw of face in radians as tuple """
+        info = self._info
+        roll = 0 if info["roll"] is None else info["roll"]
+        pitch = 0 if info["pitch"] is None else info["pitch"]
+        yaw = 0 if info["yaw"] is None else info["yaw"]
+
+        return (roll, pitch, yaw)
+
+    @property
+    def roll(self):
+        """ Return roll angle in radians of the face region """
+        roll, _, _ = self.roll_pitch_yaw()
+        return roll
+
+    @property
+    def pitch(self):
+        """ Return pitch angle in radians of the face region """
+        _, pitch, _ = self.roll_pitch_yaw()
+        return pitch
+
+    @property
+    def yaw(self):
+        """ Return yaw angle in radians of the face region """
+        _, _, yaw = self.roll_pitch_yaw()
+        return yaw
+
+    def _make_point(self, xy):
+        """ Translate an (x, y) tuple based on image orientation
+            and convert to image coordinates
+
+        Arguments:
+            xy: tuple of (x, y) coordinates for point to translate
+                in format used by Photos (percent of height/width)
+        
+        Returns:
+            (x, y) tuple of translated coordinates in pixels in PIL format/reference frame
+        """
+        # Reference: https://github.com/neilpa/phace/blob/7594776480505d0c389688a42099c94ac5d34f3f/cmd/phace/draw.go#L79-L94
+
+        orientation = self.photo.orientation
+        x, y = xy
+        dx = self.photo.width
+        dy = self.photo.height
+        if orientation in [1, 2]:
+            y = 1.0 - y
+        elif orientation in [3, 4]:
+            x = 1.0 - x
+        elif orientation in [5, 6]:
+            x, y = 1.0 - y, 1.0 - x
+            dx, dy = dy, dx
+        elif orientation in [7, 8]:
+            x, y = y, x
+            dx, dy = dy, dx
+        else:
+            logging.warning(f"Unhandled orientation: {orientation}")
+
+        return (int(x * dx), int(y * dy))
+
+    def _make_point_with_rotation(self, xy):
+        """ Translate an (x, y) tuple based on image orientation and rotation
+            and convert to image coordinates
+
+        Arguments:
+            xy: tuple of (x, y) coordinates for point to translate
+                in format used by Photos (percent of height/width)
+        
+        Returns:
+            (x, y) tuple of translated coordinates in pixels in PIL format/reference frame
+        """
+
+        # convert to image coordinates
+        x, y = self._make_point(xy)
+
+        # rotate about center
+        xmid, ymid = self.center
+        roll, _, _ = self.roll_pitch_yaw()
+        xr, yr = rotate_image_point(x, y, xmid, ymid, roll)
+
+        return (int(xr), int(yr))
+
+    def asdict(self):
+        """ Returns dict representation of class instance """
+        roll, pitch, yaw = self.roll_pitch_yaw()
+        return {
+            "_pk": self._pk,
+            "uuid": self.uuid,
+            "name": self.name,
+            "asset_uuid": self.asset_uuid,
+            "_person_pk": self._person_pk,
+            "center_x": self.center_x,
+            "center_y": self.center_y,
+            "center": self.center,
+            "mouth_x": self.mouth_x,
+            "mouth_y": self.mouth_y,
+            "mouth": self.mouth,
+            "left_eye_x": self.left_eye_x,
+            "left_eye_y": self.left_eye_y,
+            "left_eye": self.left_eye,
+            "right_eye_x": self.right_eye_x,
+            "right_eye_y": self.right_eye_y,
+            "right_eye": self.right_eye,
+            "size": self.size,
+            "face_rect": self.face_rect(),
+            "roll": roll,
+            "pitch": pitch,
+            "yaw": yaw,
+            "quality": self.quality,
+            "source_width": self.source_width,
+            "source_height": self.source_height,
+            "has_smile": self.has_smile,
+            "left_eye_closed": self.left_eye_closed,
+            "right_eye_closed": self.right_eye_closed,
+            "manual": self.manual,
+            "face_type": self.face_type,
+            "age_type": self.age_type,
+            "bald_type": self.bald_type,
+            "eye_makeup_type": self.eye_makeup_type,
+            "eye_state": self.eye_state,
+            "facial_hair_type": self.facial_hair_type,
+            "gender_type": self.gender_type,
+            "glasses_type": self.glasses_type,
+            "hair_color_type": self.hair_color_type,
+            "intrash": self.intrash,
+            "lip_makeup_type": self.lip_makeup_type,
+            "smile_type": self.smile_type,
+        }
+
+    def json(self):
+        """ Return JSON representation of FaceInfo instance """
+        return json.dumps(self.asdict())
+
+    def __str__(self):
+        return f"FaceInfo(uuid={self.uuid}, center_x={self.center_x}, center_y = {self.center_y}, size={self.size}, person={self.name}, asset_uuid={self.asset_uuid})"
+
+    def __repr__(self):
+        return f"FaceInfo(db={self._db}, pk={self._pk})"
+
+    def __eq__(self, other):
+        if not isinstance(other, type(self)):
+            return False
+
+        return all(
+            getattr(self, field) == getattr(other, field) for field in ["_db", "_pk"]
+        )
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+
+def rotate_image_point(x, y, xmid, ymid, angle):
+    """ rotate image point about xm, ym by angle in radians
+
+    Arguments:
+        x: x coordinate of point to rotate    
+        y: y coordinate of point to rotate
+        xmid: x coordinate of center point to rotate about
+        ymid: y coordinate of center point to rotate about
+        angle: angle in radians about which to coordinate, 
+            counter-clockwise is positive
+
+    Returns:
+        tuple of rotated points (xr, yr)
+    """
+    # translate point relative to the mid point
+    x = x - xmid
+    y = y - ymid
+
+    # rotate by angle and translate back
+    # the photo coordinate system is downwards y is positive so
+    # need to adjust the rotation accordingly
+    cos_angle = math.cos(angle)
+    sin_angle = math.sin(angle)
+    xr = x * cos_angle + y * sin_angle + xmid
+    yr = -x * sin_angle + y * cos_angle + ymid
+
+    return (xr, yr)

osxphotos/photoinfo/__init__.py

@@ -6,4 +6,5 @@ PhotosDB.photos() returns a list of PhotoInfo objects
 
 from ._photoinfo_exifinfo import ExifInfo
 from ._photoinfo_export import ExportResults
+from ._photoinfo_scoreinfo import ScoreInfo
 from .photoinfo import PhotoInfo

osxphotos/photoinfo/_photoinfo_comments.py

@@ -0,0 +1,17 @@
+""" PhotoInfo methods to expose comments and likes for shared photos """ 
+
+@property
+def comments(self):
+    """ Returns list of Comment objects for any comments on the photo (sorted by date) """
+    try:
+        return self._db._db_comments_uuid[self.uuid]["comments"]
+    except:
+        return []
+
+@property
+def likes(self):
+    """ Returns list of Like objects for any likes on the photo (sorted by date) """
+    try:
+        return self._db._db_comments_uuid[self.uuid]["likes"]
+    except:
+        return []

osxphotos/photoinfo/_photoinfo_exiftool.py

@@ -5,6 +5,7 @@ import os
 
 from ..exiftool import ExifTool, get_exiftool_path
 
+
 @property
 def exiftool(self):
     """ Returns an ExifTool object for the photo
@@ -26,8 +27,9 @@ def exiftool(self):
         except FileNotFoundError:
             # get_exiftool_path raises FileNotFoundError if exiftool not found
             exiftool = None
-            logging.warning(f"exiftool not in path; download and install from https://exiftool.org/")
+            logging.warning(
+                f"exiftool not in path; download and install from https://exiftool.org/"
+            )
 
         self._exiftool = exiftool
         return self._exiftool
-

osxphotos/photoinfo/_photoinfo_export.py

@@ -11,7 +11,6 @@
 
 # TODO: should this be its own PhotoExporter class?
 
-import filecmp
 import glob
 import json
 import logging
@@ -31,13 +30,14 @@ from .._constants import (
     _UNKNOWN_PERSON,
     _XMP_TEMPLATE_NAME,
 )
-from .._export_db import ExportDBNoOp
+from ..export_db import ExportDBNoOp
 from ..exiftool import ExifTool
 from ..fileutil import FileUtil
-from ..utils import dd_to_dms_str
+from ..utils import dd_to_dms_str, findfiles
 
 ExportResults = namedtuple(
-    "ExportResults", ["exported", "new", "updated", "skipped", "exif_updated"]
+    "ExportResults",
+    ["exported", "new", "updated", "skipped", "exif_updated", "touched"],
 )
 
 
@@ -106,7 +106,7 @@ def _export_photo_uuid_applescript(
     )
 
     dest = pathlib.Path(dest)
-    if not dest.is_dir:
+    if not dest.is_dir():
         raise ValueError(f"dest {dest} must be a directory")
 
     if not original ^ edited:
@@ -215,6 +215,7 @@ def export(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
 ):
     """ export photo 
         dest: must be valid destination path (or exception raised) 
@@ -250,6 +251,7 @@ def export(
         use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords
         when exporting metadata with exiftool or sidecar
         keyword_template: (list of strings); list of template strings that will be rendered as used as keywords
+        description_template: string; optional template string that will be rendered for use as photo description
         returns: list of photos exported
         """
 
@@ -273,6 +275,7 @@ def export(
         use_albums_as_keywords=use_albums_as_keywords,
         use_persons_as_keywords=use_persons_as_keywords,
         keyword_template=keyword_template,
+        description_template=description_template,
     )
 
     return results.exported
@@ -297,10 +300,14 @@ def export2(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
     update=False,
     export_db=None,
     fileutil=FileUtil,
     dry_run=False,
+    touch_file=False,
+    convert_to_jpeg=False,
+    jpeg_quality=1.0,
 ):
     """ export photo, like export but with update and dry_run options
         dest: must be valid destination path or exception raised 
@@ -308,10 +315,8 @@ def export2(
                     **NOTE**: if provided, user must ensure file extension (suffix) is correct. 
                     For example, if photo is .CR2 file, edited image may be .jpeg.  
                     If you provide an extension different than what the actual file is, 
-                    export will print a warning but will export the photo using the 
+                    will export the photo using the incorrect file extension (unless use_photos_export is true, 
-                    incorrect file extension (unless use_photos_export is true, in which case export will
+                    in which case export will use the extension provided by Photos upon export.
-                    use the extension provided by Photos upon export; in this case, an incorrect extension is
-                    silently ignored).
                     e.g. to get the extension of the edited photo, 
                     reference PhotoInfo.path_edited
         edited: (boolean, default=False); if True will export the edited version of the photo 
@@ -330,18 +335,21 @@ def export2(
         timeout: (int, default=120) timeout in seconds used with use_photos_export
         exiftool: (boolean, default = False); if True, will use exiftool to write metadata to export file
         no_xattr: (boolean, default = False); if True, exports file without preserving extended attributes
-        returns list of full paths to the exported files
         use_albums_as_keywords: (boolean, default = False); if True, will include album names in keywords
         when exporting metadata with exiftool or sidecar
         use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords
         when exporting metadata with exiftool or sidecar
         keyword_template: (list of strings); list of template strings that will be rendered as used as keywords
+        description_template: string; optional template string that will be rendered for use as photo description
         update: (boolean, default=False); if True export will run in update mode, that is, it will
                 not export the photo if the current version already exists in the destination
         export_db: (ExportDB_ABC); instance of a class that conforms to ExportDB_ABC with methods
                 for getting/setting data related to exported files to compare update state
         fileutil: (FileUtilABC); class that conforms to FileUtilABC with various file utilities
         dry_run: (boolean, default=False); set to True to run in "dry run" mode
+        touch_file: (boolean, default=False); if True, sets file's modification time upon photo date
+        convert_to_jpeg: boolean; if True, converts non-jpeg images to jpeg
+        jpeg_quality: float in range 0.0 <= jpeg_quality <= 1.0.  A value of 1.0 specifies use best quality, a value of 0.0 specifies use maximum compression.
 
         Returns: ExportResults namedtuple with fields: exported, new, updated, skipped 
                     where each field is a list of file paths
@@ -350,6 +358,10 @@ def export2(
               and no-op fileutil (e.g. ExportDBInMemory and FileUtilNoOp)
             """
 
+    # NOTE: This function is very complex and does a lot of things.
+    # Don't modify this code if you don't fully understand everything it does.
+    # TODO: This is a good candidate for refactoring.
+
     # when called from export(), won't get an export_db, so use no-op version
     if export_db is None:
         export_db = ExportDBNoOp()
@@ -370,6 +382,9 @@ def export2(
     # list of all files skipped because they do not need to be updated (for use with update=True)
     update_skipped_files = []
 
+    # list of all files with utime touched (touch_file = True)
+    touched_files = []
+
     # check edited and raise exception trying to export edited version of
     # photo that hasn't been edited
     if edited and not self.hasadjustments:
@@ -382,34 +397,41 @@ def export2(
         raise TypeError(
             "Too many positional arguments.  Should be at most two: destination, filename."
         )
-    else:
-        # verify destination is a valid path
-        if dest is None:
-            raise ValueError("Destination must not be None")
-        elif not dry_run and not os.path.isdir(dest):
-            raise FileNotFoundError("Invalid path passed to export")
 
-        if filename and len(filename) == 1:
+    # verify destination is a valid path
-            # if filename passed, use it
+    if dest is None:
-            fname = filename[0]
+        raise ValueError("Destination must not be None")
-        else:
+    elif not dry_run and not os.path.isdir(dest):
-            # no filename provided so use the default
+        raise FileNotFoundError("Invalid path passed to export")
-            # if edited file requested, use filename but add _edited
+
-            # need to use file extension from edited file as Photos saves a jpeg once edited
+    if filename and len(filename) == 1:
-            if edited and not use_photos_export:
+        # if filename passed, use it
-                # verify we have a valid path_edited and use that to get filename
+        fname = filename[0]
-                if not self.path_edited:
+    else:
-                    raise FileNotFoundError(
+        # no filename provided so use the default
-                        "edited=True but path_edited is none; hasadjustments: "
+        # if edited file requested, use filename but add _edited
-                        f" {self.hasadjustments}"
+        # need to use file extension from edited file as Photos saves a jpeg once edited
-                    )
+        if edited and not use_photos_export:
-                edited_name = pathlib.Path(self.path_edited).name
+            # verify we have a valid path_edited and use that to get filename
-                edited_suffix = pathlib.Path(edited_name).suffix
+            if not self.path_edited:
-                fname = (
+                raise FileNotFoundError(
-                    pathlib.Path(self.filename).stem + edited_identifier + edited_suffix
+                    "edited=True but path_edited is none; hasadjustments: "
+                    f" {self.hasadjustments}"
                 )
-            else:
+            edited_name = pathlib.Path(self.path_edited).name
-                fname = self.filename
+            edited_suffix = pathlib.Path(edited_name).suffix
+            fname = pathlib.Path(self.filename).stem + edited_identifier + edited_suffix
+        else:
+            fname = self.filename
+
+    uti = self.uti if edited else self.uti_original
+    if convert_to_jpeg and self.isphoto and uti != "public.jpeg":
+        # not a jpeg but will convert to jpeg upon export so fix file extension
+        fname_new = pathlib.Path(fname)
+        fname = str(fname_new.parent / f"{fname_new.stem}.jpeg")
+    else:
+        # nothing to convert
+        convert_to_jpeg = False
 
     # check destination path
     dest = pathlib.Path(dest)
@@ -423,11 +445,10 @@ def export2(
     # dest will be file1 (1).jpeg even though file1.jpeg doesn't exist to prevent sidecar collision
     if not update and increment and not overwrite:
         count = 1
-        glob_str = str(dest.parent / f"{dest.stem}*")
+        dest_files = findfiles(f"{dest.stem}*", str(dest.parent))
-        dest_files = glob.glob(glob_str)
+        dest_files = [pathlib.Path(f).stem.lower() for f in dest_files]
-        dest_files = [pathlib.Path(f).stem for f in dest_files]
         dest_new = dest.stem
-        while dest_new in dest_files:
+        while dest_new.lower() in dest_files:
             dest_new = f"{dest.stem} ({count})"
             count += 1
         dest = dest.parent / f"{dest_new}{dest.suffix}"
@@ -464,21 +485,17 @@ def export2(
             raise FileNotFoundError(f"{src} does not appear to exist")
 
         if not _check_export_suffix(src, dest, edited):
-            logging.warning(
+            logging.debug(
                 f"Invalid destination suffix: {dest.suffix} for {self.path}, "
                 + f"edited={edited}, path_edited={self.path_edited}, "
                 + f"original_filename={self.original_filename}, filename={self.filename}"
             )
 
-        logging.debug(
-            f"exporting {src} to {dest}, overwrite={overwrite}, increment={increment}, dest exists: {dest.exists()}"
-        )
-
         # found source now try to find right destination
         if update and dest.exists():
             # destination exists, check to see if destination is the right UUID
             dest_uuid = export_db.get_uuid_for_file(dest)
-            if dest_uuid is None and filecmp.cmp(src, dest):
+            if dest_uuid is None and fileutil.cmp(src, dest):
                 # might be exporting into a pre-ExportDB folder or the DB got deleted
                 logging.debug(
                     f"Found matching file with blank uuid: {self.uuid}, {dest}"
@@ -489,14 +506,13 @@ def export2(
                     self.uuid,
                     fileutil.file_sig(dest),
                     (None, None, None),
+                    (None, None, None),
+                    (None, None, None),
                     self.json(),
                     None,
                 )
             if dest_uuid != self.uuid:
                 # not the right file, find the right one
-                logging.debug(
-                    f"Need to find right photo: uuid={self.uuid}, dest={dest_uuid}, dest={dest}, path={self.path}"
-                )
                 count = 1
                 glob_str = str(dest.parent / f"{dest.stem} (*{dest.suffix}")
                 dest_files = glob.glob(glob_str)
@@ -504,17 +520,11 @@ def export2(
                 for file_ in dest_files:
                     dest_uuid = export_db.get_uuid_for_file(file_)
                     if dest_uuid == self.uuid:
-                        logging.debug(
-                            f"Found matching file for uuid: {dest_uuid}, {file_}"
-                        )
                         dest = pathlib.Path(file_)
                         found_match = True
                         break
-                    elif dest_uuid is None and filecmp.cmp(src, file_):
+                    elif dest_uuid is None and fileutil.cmp(src, file_):
                         # files match, update the UUID
-                        logging.debug(
-                            f"Found matching file with blank uuid: {self.uuid}, {file_}"
-                        )
                         dest = pathlib.Path(file_)
                         found_match = True
                         export_db.set_data(
@@ -522,16 +532,14 @@ def export2(
                             self.uuid,
                             fileutil.file_sig(dest),
                             (None, None, None),
+                            (None, None, None),
+                            (None, None, None),
                             self.json(),
                             None,
                         )
                         break
 
                 if not found_match:
-                    logging.debug(
-                        f"Didn't find destination match for uuid {self.uuid} {dest}"
-                    )
-
                     # increment the destination file
                     count = 1
                     glob_str = str(dest.parent / f"{dest.stem}*")
@@ -542,7 +550,6 @@ def export2(
                         dest_new = f"{dest.stem} ({count})"
                         count += 1
                     dest = dest.parent / f"{dest_new}{dest.suffix}"
-                    logging.debug(f"New destination = {dest}, uuid = {self.uuid}")
 
         # export the dest file
         results = self._export_photo(
@@ -554,12 +561,17 @@ def export2(
             no_xattr,
             export_as_hardlink,
             exiftool,
-            fileutil,
+            touch_file,
+            convert_to_jpeg,
+            fileutil=fileutil,
+            edited=edited,
+            jpeg_quality=jpeg_quality,
         )
         exported_files = results.exported
         update_new_files = results.new
         update_updated_files = results.updated
         update_skipped_files = results.skipped
+        touched_files = results.touched
 
         # copy live photo associated .mov if requested
         if live_photo and self.live_photo:
@@ -579,12 +591,15 @@ def export2(
                     no_xattr,
                     export_as_hardlink,
                     exiftool,
-                    fileutil,
+                    touch_file,
+                    False,
+                    fileutil=fileutil,
                 )
                 exported_files.extend(results.exported)
                 update_new_files.extend(results.new)
                 update_updated_files.extend(results.updated)
                 update_skipped_files.extend(results.skipped)
+                touched_files.extend(results.touched)
             else:
                 logging.debug(f"Skipping missing live movie for {filename}")
 
@@ -604,27 +619,34 @@ def export2(
                     no_xattr,
                     export_as_hardlink,
                     exiftool,
-                    fileutil,
+                    touch_file,
+                    convert_to_jpeg,
+                    fileutil=fileutil,
+                    jpeg_quality=jpeg_quality,
                 )
                 exported_files.extend(results.exported)
                 update_new_files.extend(results.new)
                 update_updated_files.extend(results.updated)
                 update_skipped_files.extend(results.skipped)
+                touched_files.extend(results.touched)
             else:
                 logging.debug(f"Skipping missing RAW photo for {filename}")
     else:
         # use_photo_export
-        exported = None
+        exported = []
         # export live_photo .mov file?
         live_photo = True if live_photo and self.live_photo else False
-        if edited:
+        if edited or self.shared:
             # exported edited version and not original
+            # shared photos (in shared albums) show up as not having adjustments (not edited)
+            # but Photos is unable to export the "original" as only a jpeg copy is shared in iCloud
+            # so tell Photos to export the current version in this case
             if filename:
                 # use filename stem provided
                 filestem = dest.stem
             else:
                 # didn't get passed a filename, add _edited
-                filestem = f"{dest.stem}_edited"
+                filestem = f"{dest.stem}{edited_identifier}"
                 dest = dest.parent / f"{filestem}.jpeg"
 
             exported = _export_photo_uuid_applescript(
@@ -652,24 +674,30 @@ def export2(
                 burst=self.burst,
                 dry_run=dry_run,
             )
-
+        if exported:
-        if exported is not None:
+            if touch_file:
+                for exported_file in exported:
+                    touched_files.append(exported_file)
+                    ts = int(self.date.timestamp())
+                    fileutil.utime(exported_file, (ts, ts))
             exported_files.extend(exported)
+            if update:
+                update_new_files.extend(exported)
+
         else:
             logging.warning(
                 f"Error exporting photo {self.uuid} to {dest} with use_photos_export"
             )
 
     # export metadata
-    info = export_db.get_info_for_uuid(self.uuid)
-
     if sidecar_json:
         logging.debug("writing exiftool_json_sidecar")
-        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.json")
+        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}{dest.suffix}.json")
         sidecar_str = self._exiftool_json_sidecar(
             use_albums_as_keywords=use_albums_as_keywords,
             use_persons_as_keywords=use_persons_as_keywords,
             keyword_template=keyword_template,
+            description_template=description_template,
         )
         if not dry_run:
             try:
@@ -680,11 +708,13 @@ def export2(
 
     if sidecar_xmp:
         logging.debug("writing xmp_sidecar")
-        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.xmp")
+        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}{dest.suffix}.xmp")
         sidecar_str = self._xmp_sidecar(
             use_albums_as_keywords=use_albums_as_keywords,
             use_persons_as_keywords=use_persons_as_keywords,
             keyword_template=keyword_template,
+            description_template=description_template,
+            extension=dest.suffix[1:] if dest.suffix else None,
         )
         if not dry_run:
             try:
@@ -712,6 +742,7 @@ def export2(
                         use_albums_as_keywords=use_albums_as_keywords,
                         use_persons_as_keywords=use_persons_as_keywords,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                     )
                 )[0]
                 if old_data != current_data:
@@ -720,13 +751,13 @@ def export2(
             if old_data is None or files_are_different:
                 # didn't have old data, assume we need to write it
                 # or files were different
-                logging.debug(f"No exifdata for {exported_file}, writing it")
                 if not dry_run:
                     self._write_exif_data(
                         exported_file,
                         use_albums_as_keywords=use_albums_as_keywords,
                         use_persons_as_keywords=use_persons_as_keywords,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                     )
                 export_db.set_exifdata_for_file(
                     exported_file,
@@ -734,6 +765,7 @@ def export2(
                         use_albums_as_keywords=use_albums_as_keywords,
                         use_persons_as_keywords=use_persons_as_keywords,
                         keyword_template=keyword_template,
+                        description_template=description_template,
                     ),
                 )
                 export_db.set_stat_exif_for_file(
@@ -742,20 +774,22 @@ def export2(
                 exif_files_updated.append(exported_file)
     elif exiftool and exif_files:
         for exported_file in exif_files:
-            logging.debug(f"Writing exif data to {exported_file}")
             if not dry_run:
                 self._write_exif_data(
                     exported_file,
                     use_albums_as_keywords=use_albums_as_keywords,
                     use_persons_as_keywords=use_persons_as_keywords,
                     keyword_template=keyword_template,
+                    description_template=description_template,
                 )
+
             export_db.set_exifdata_for_file(
                 exported_file,
                 self._exiftool_json_sidecar(
                     use_albums_as_keywords=use_albums_as_keywords,
                     use_persons_as_keywords=use_persons_as_keywords,
                     keyword_template=keyword_template,
+                    description_template=description_template,
                 ),
             )
             export_db.set_stat_exif_for_file(
@@ -763,13 +797,23 @@ def export2(
             )
             exif_files_updated.append(exported_file)
 
-    return ExportResults(
+    if touch_file:
+        for exif_file in exif_files_updated:
+            touched_files.append(exif_file)
+            ts = int(self.date.timestamp())
+            fileutil.utime(exif_file, (ts, ts))
+
+    touched_files = list(set(touched_files))
+
+    results = ExportResults(
         exported_files,
         update_new_files,
         update_updated_files,
         update_skipped_files,
         exif_files_updated,
+        touched_files,
     )
+    return results
 
 
 def _export_photo(
@@ -782,11 +826,15 @@ def _export_photo(
     no_xattr,
     export_as_hardlink,
     exiftool,
+    touch_file,
+    convert_to_jpeg,
     fileutil=FileUtil,
+    edited=False,
+    jpeg_quality=1.0,
 ):
     """ Helper function for export()
         Does the actual copy or hardlink taking the appropriate 
-        action depending on update, overwrite
+        action depending on update, overwrite, export_as_hardlink
         Assumes destination is the right destination (e.g. UUID matches)
         sets UUID and JSON info foo exported file using set_uuid_for_file, set_inf_for_uuido
     
@@ -799,153 +847,146 @@ def _export_photo(
         no_xattr: don't copy extended attributes
         export_as_hardlink: bool
         exiftool: bool
+        touch_file: bool
+        convert_to_jpeg: bool; if True, convert file to jpeg on export
         fileutil: FileUtil class that conforms to fileutil.FileUtilABC
+        edited: bool; set to True if exporting edited version of photo
+        jpeg_quality: float in range 0.0 <= jpeg_quality <= 1.0.  A value of 1.0 specifies use best quality, a value of 0.0 specifies use maximum compression.
 
     Returns:
         ExportResults
+
+    Raises:
+        ValueError if export_as_hardlink and convert_to_jpeg both True
     """
 
+    if export_as_hardlink and convert_to_jpeg:
+        raise ValueError("export_as_hardlink and convert_to_jpeg cannot both be True")
+
     exported_files = []
     update_updated_files = []
     update_new_files = []
     update_skipped_files = []
+    touched_files = []
 
     dest_str = str(dest)
     dest_exists = dest.exists()
-    if export_as_hardlink:
+    op_desc = "export_as_hardlink" if export_as_hardlink else "export_by_copying"
-        # use hardlink instead of copy
+
-        if not update:
+    if update:  # updating
-            # not update, do the the hardlink
+        cmp_touch, cmp_orig = False, False
-            if overwrite and dest.exists():
+        if dest_exists:
-                # need to remove the destination first
+            # update, destination exists, but we might not need to replace it...
-                # dest.unlink()
+            if exiftool:
-                fileutil.unlink(dest)
+                sig_exif = export_db.get_stat_exif_for_file(dest_str)
-            logging.debug(f"Not update: export_as_hardlink linking file {src} {dest}")
+                cmp_orig = fileutil.cmp_file_sig(dest_str, sig_exif)
-            fileutil.hardlink(src, dest)
+                sig_exif = (sig_exif[0], sig_exif[1], int(self.date.timestamp()))
-            export_db.set_data(
+                cmp_touch = fileutil.cmp_file_sig(dest_str, sig_exif)
-                dest_str,
+            elif convert_to_jpeg:
-                self.uuid,
+                sig_converted = export_db.get_stat_converted_for_file(dest_str)
-                fileutil.file_sig(dest_str),
+                cmp_orig = fileutil.cmp_file_sig(dest_str, sig_converted)
-                (None, None, None),
+                sig_converted = (
-                self.json(),
+                    sig_converted[0],
-                None,
+                    sig_converted[1],
-            )
+                    int(self.date.timestamp()),
-            exported_files.append(dest_str)
+                )
-        elif dest_exists and dest.samefile(src):
+                cmp_touch = fileutil.cmp_file_sig(dest_str, sig_converted)
-            # update, hardlink and it already points to the right file, do nothing
+            else:
-            logging.debug(
+                cmp_orig = fileutil.cmp(src, dest)
-                f"Update: skipping samefile with export_as_hardlink {src} {dest}"
+                cmp_touch = fileutil.cmp(src, dest, mtime1=int(self.date.timestamp()))
-            )
+
-            update_skipped_files.append(dest_str)
+            sig_cmp = cmp_touch if touch_file else cmp_orig
-        elif dest_exists:
+
-            # update, not the same file (e.g. user may not have used export_as_hardlink last time it was run
+            if edited:
-            logging.debug(
+                # requested edited version of photo
-                f"Update: removing existing file prior to export_as_hardlink {src} {dest}"
+                # need to see if edited version in Photos library has changed
-            )
+                # (e.g. it's been edited again)
-            # dest.unlink()
+                sig_edited = export_db.get_stat_edited_for_file(dest_str)
-            fileutil.unlink(dest)
+                cmp_edited = (
-            fileutil.hardlink(src, dest)
+                    fileutil.cmp_file_sig(src, sig_edited)
-            export_db.set_data(
+                    if sig_edited != (None, None, None)
-                dest_str,
+                    else False
-                self.uuid,
+                )
-                fileutil.file_sig(dest_str),
+                sig_cmp = sig_cmp and cmp_edited
-                (None, None, None),
+
-                self.json(),
+            if (export_as_hardlink and dest.samefile(src)) or (
-                None,
+                not export_as_hardlink and not dest.samefile(src) and sig_cmp
-            )
+            ):
-            update_updated_files.append(dest_str)
+                # destination exists and signatures match, skip it
-            exported_files.append(dest_str)
+                update_skipped_files.append(dest_str)
+            else:
+                # destination exists but signature is different
+                if touch_file and cmp_orig and not cmp_touch:
+                    # destination exists, signature matches original but does not match expected touch time
+                    # skip exporting but update touch time
+                    update_skipped_files.append(dest_str)
+                    touched_files.append(dest_str)
+                elif not touch_file and cmp_touch and not cmp_orig:
+                    # destination exists, signature matches expected touch but not original
+                    # user likely exported with touch_file and is now exporting without touch_file
+                    # don't update the file because it's same but leave touch time
+                    update_skipped_files.append(dest_str)
+                else:
+                    # destination exists but is different
+                    update_updated_files.append(dest_str)
+                    if touch_file:
+                        touched_files.append(dest_str)
+
         else:
-            # update, hardlink, destination doesn't exist (new file)
+            # update, destination doesn't exist (new file)
-            logging.debug(
+            logging.debug(f"Update: exporting new file with {op_desc} {src} {dest}")
-                f"Update: exporting new file with export_as_hardlink {src} {dest}"
-            )
-            fileutil.hardlink(src, dest)
-            export_db.set_data(
-                dest_str,
-                self.uuid,
-                fileutil.file_sig(dest_str),
-                (None, None, None),
-                self.json(),
-                None,
-            )
-            exported_files.append(dest_str)
             update_new_files.append(dest_str)
+            if touch_file:
+                touched_files.append(dest_str)
     else:
-        if not update:
+        # not update, export the file
-            # not update, do the the copy
+        logging.debug(f"Exporting file with {op_desc} {src} {dest}")
-            if overwrite and dest.exists():
+        exported_files.append(dest_str)
-                # need to remove the destination first
+        if touch_file:
-                # dest.unlink()
+            sig = fileutil.file_sig(src)
-                fileutil.unlink(dest)
+            sig = (sig[0], sig[1], int(self.date.timestamp()))
-            logging.debug(f"Not update: copying file {src} {dest}")
+            if not fileutil.cmp_file_sig(src, sig):
-            fileutil.copy(src, dest_str, norsrc=no_xattr)
+                touched_files.append(dest_str)
-            exported_files.append(dest_str)
+    if not update_skipped_files:
-            export_db.set_data(
+        converted_stat = (None, None, None)
-                dest_str,
+        edited_stat = fileutil.file_sig(src) if edited else (None, None, None)
-                self.uuid,
+        if dest_exists and (update or overwrite):
-                fileutil.file_sig(dest_str),
+            # need to remove the destination first
-                (None, None, None),
+            logging.debug(
-                self.json(),
+                f"Update: removing existing file prior to {op_desc} {src} {dest}"
-                None,
             )
-        # elif dest_exists and not exiftool and cmp_file(dest_str, export_db.get_stat_orig_for_file(dest_str)):
-        elif (
-            dest_exists
-            and not exiftool
-            and filecmp.cmp(src, dest)
-            and not dest.samefile(src)
-        ):
-            # destination exists but is identical
-            logging.debug(f"Update: skipping identifical original files {src} {dest}")
-            # call set_stat because code can reach this spot if no export DB but exporting a RAW or live photo
-            # potentially re-writes the data in the database but ensures database is complete
-            export_db.set_stat_orig_for_file(dest_str, fileutil.file_sig(dest_str))
-            update_skipped_files.append(dest_str)
-        elif (
-            dest_exists
-            and exiftool
-            and fileutil.cmp_sig(dest_str, export_db.get_stat_exif_for_file(dest_str))
-            and not dest.samefile(src)
-        ):
-            # destination exists but is identical
-            logging.debug(f"Update: skipping identifical exiftool files {src} {dest}")
-            update_skipped_files.append(dest_str)
-        elif dest_exists:
-            # destination exists but is different or is a hardlink
-            logging.debug(f"Update: removing existing file prior to copy {src} {dest}")
-            stat_src = os.stat(src)
-            stat_dest = os.stat(dest)
-            # dest.unlink()
             fileutil.unlink(dest)
-            fileutil.copy(src, dest_str, norsrc=no_xattr)
+        if export_as_hardlink:
-            export_db.set_data(
+            fileutil.hardlink(src, dest)
-                dest_str,
+        elif convert_to_jpeg:
-                self.uuid,
+            # use convert_to_jpeg to export the file
-                fileutil.file_sig(dest_str),
+            fileutil.convert_to_jpeg(src, dest_str, compression_quality=jpeg_quality)
-                (None, None, None),
+            converted_stat = fileutil.file_sig(dest_str)
-                self.json(),
-                None,
-            )
-            exported_files.append(dest_str)
-            update_updated_files.append(dest_str)
         else:
-            # destination doesn't exist, copy the file
-            logging.debug(f"Update: copying new file {src} {dest}")
             fileutil.copy(src, dest_str, norsrc=no_xattr)
-            export_db.set_data(
+
-                dest_str,
+        export_db.set_data(
-                self.uuid,
+            dest_str,
-                fileutil.file_sig(dest_str),
+            self.uuid,
-                (None, None, None),
+            fileutil.file_sig(dest_str),
-                self.json(),
+            (None, None, None),
-                None,
+            converted_stat,
-            )
+            edited_stat,
-            exported_files.append(dest_str)
+            self.json(),
-            update_new_files.append(dest_str)
+            None,
+        )
+
+    if touched_files:
+        ts = int(self.date.timestamp())
+        fileutil.utime(dest, (ts, ts))
 
     return ExportResults(
-        exported_files, update_new_files, update_updated_files, update_skipped_files, []
+        exported_files + update_new_files + update_updated_files,
+        update_new_files,
+        update_updated_files,
+        update_skipped_files,
+        [],
+        touched_files,
     )
 
 
@@ -955,6 +996,7 @@ def _write_exif_data(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
 ):
     """ write exif data to image file at filepath
     filepath: full path to the image file """
@@ -966,6 +1008,7 @@ def _write_exif_data(
             use_albums_as_keywords=use_albums_as_keywords,
             use_persons_as_keywords=use_persons_as_keywords,
             keyword_template=keyword_template,
+            description_template=description_template,
         )
     )[0]
     for exiftag, val in exif_info.items():
@@ -984,6 +1027,7 @@ def _exiftool_json_sidecar(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
 ):
     """ return json string of EXIF details in exiftool sidecar format
         Does not include all the EXIF fields as those are likely already in the image
@@ -1009,7 +1053,13 @@ def _exiftool_json_sidecar(
     exif = {}
     exif["_CreatedBy"] = "osxphotos, https://github.com/RhetTbull/osxphotos"
 
-    if self.description:
+    if description_template is not None:
+        description = self.render_template(
+            description_template, expand_inplace=True, inplace_sep=", "
+        )[0]
+        exif["EXIF:ImageDescription"] = description
+        exif["XMP:Description"] = description
+    elif self.description:
         exif["EXIF:ImageDescription"] = self.description
         exif["XMP:Description"] = self.description
 
@@ -1079,12 +1129,10 @@ def _exiftool_json_sidecar(
 
     (lat, lon) = self.location
     if lat is not None and lon is not None:
-        lat_str, lon_str = dd_to_dms_str(lat, lon)
+        exif["EXIF:GPSLatitude"] = lat
-        exif["EXIF:GPSLatitude"] = lat_str
+        exif["EXIF:GPSLongitude"] = lon
-        exif["EXIF:GPSLongitude"] = lon_str
+        lat_ref = "N" if lat >= 0 else "S"
-        exif["Composite:GPSPosition"] = f"{lat_str}, {lon_str}"
+        lon_ref = "E" if lon >= 0 else "W"
-        lat_ref = "North" if lat >= 0 else "South"
-        lon_ref = "East" if lon >= 0 else "West"
         exif["EXIF:GPSLatitudeRef"] = lat_ref
         exif["EXIF:GPSLongitudeRef"] = lon_ref
 
@@ -1112,16 +1160,28 @@ def _xmp_sidecar(
     use_albums_as_keywords=False,
     use_persons_as_keywords=False,
     keyword_template=None,
+    description_template=None,
+    extension=None,
 ):
     """ returns string for XMP sidecar 
         use_albums_as_keywords: treat album names as keywords
         use_persons_as_keywords: treat person names as keywords
-        keyword_template: (list of strings); list of template strings to render as keywords """
+        keyword_template: (list of strings); list of template strings to render as keywords 
-
+        description_template: string; optional template string that will be rendered for use as photo description """
-    # TODO: add additional fields to XMP file?
 
     xmp_template = Template(filename=os.path.join(_TEMPLATE_DIR, _XMP_TEMPLATE_NAME))
 
+    if extension is None:
+        extension = pathlib.Path(self.original_filename)
+        extension = extension.suffix[1:] if extension.suffix else None
+
+    if description_template is not None:
+        description = self.render_template(
+            description_template, expand_inplace=True, inplace_sep=", "
+        )[0]
+    else:
+        description = self.description if self.description is not None else ""
+
     keyword_list = []
     if self.keywords:
         keyword_list.extend(self.keywords)
@@ -1178,7 +1238,12 @@ def _xmp_sidecar(
         subject_list = list(self.keywords) + person_list
 
     xmp_str = xmp_template.render(
-        photo=self, keywords=keyword_list, persons=person_list, subjects=subject_list
+        photo=self,
+        description=description,
+        keywords=keyword_list,
+        persons=person_list,
+        subjects=subject_list,
+        extension=extension,
     )
 
     # remove extra lines that mako inserts from template

osxphotos/photoinfo/_photoinfo_scoreinfo.py

@@ -0,0 +1,119 @@
+""" PhotoInfo methods to expose computed score info from the library """
+
+import logging
+from dataclasses import dataclass
+
+from .._constants import _PHOTOS_4_VERSION
+
+
+@dataclass(frozen=True)
+class ScoreInfo:
+    """ Computed photo score info associated with a photo from the Photos library """
+
+    overall: float
+    curation: float
+    promotion: float
+    highlight_visibility: float
+    behavioral: float
+    failure: float
+    harmonious_color: float
+    immersiveness: float
+    interaction: float
+    interesting_subject: float
+    intrusive_object_presence: float
+    lively_color: float
+    low_light: float
+    noise: float
+    pleasant_camera_tilt: float
+    pleasant_composition: float
+    pleasant_lighting: float
+    pleasant_pattern: float
+    pleasant_perspective: float
+    pleasant_post_processing: float
+    pleasant_reflection: float
+    pleasant_symmetry: float
+    sharply_focused_subject: float
+    tastefully_blurred: float
+    well_chosen_subject: float
+    well_framed_subject: float
+    well_timed_shot: float
+
+
+@property
+def score(self):
+    """ Computed score information for a photo
+
+    Returns:
+        ScoreInfo instance
+    """
+
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        logging.debug(f"score not implemented for this database version")
+        return None
+
+    try:
+        return self._scoreinfo  # pylint: disable=access-member-before-definition
+    except AttributeError:
+        try:
+            scores = self._db._db_scoreinfo_uuid[self.uuid]
+            self._scoreinfo = ScoreInfo(
+                overall=scores["overall_aesthetic"],
+                curation=scores["curation"],
+                promotion=scores["promotion"],
+                highlight_visibility=scores["highlight_visibility"],
+                behavioral=scores["behavioral"],
+                failure=scores["failure"],
+                harmonious_color=scores["harmonious_color"],
+                immersiveness=scores["immersiveness"],
+                interaction=scores["interaction"],
+                interesting_subject=scores["interesting_subject"],
+                intrusive_object_presence=scores["intrusive_object_presence"],
+                lively_color=scores["lively_color"],
+                low_light=scores["low_light"],
+                noise=scores["noise"],
+                pleasant_camera_tilt=scores["pleasant_camera_tilt"],
+                pleasant_composition=scores["pleasant_composition"],
+                pleasant_lighting=scores["pleasant_lighting"],
+                pleasant_pattern=scores["pleasant_pattern"],
+                pleasant_perspective=scores["pleasant_perspective"],
+                pleasant_post_processing=scores["pleasant_post_processing"],
+                pleasant_reflection=scores["pleasant_reflection"],
+                pleasant_symmetry=scores["pleasant_symmetry"],
+                sharply_focused_subject=scores["sharply_focused_subject"],
+                tastefully_blurred=scores["tastefully_blurred"],
+                well_chosen_subject=scores["well_chosen_subject"],
+                well_framed_subject=scores["well_framed_subject"],
+                well_timed_shot=scores["well_timed_shot"],
+            )
+            return self._scoreinfo
+        except KeyError:
+            self._scoreinfo = ScoreInfo(
+                overall=0.0,
+                curation=0.0,
+                promotion=0.0,
+                highlight_visibility=0.0,
+                behavioral=0.0,
+                failure=0.0,
+                harmonious_color=0.0,
+                immersiveness=0.0,
+                interaction=0.0,
+                interesting_subject=0.0,
+                intrusive_object_presence=0.0,
+                lively_color=0.0,
+                low_light=0.0,
+                noise=0.0,
+                pleasant_camera_tilt=0.0,
+                pleasant_composition=0.0,
+                pleasant_lighting=0.0,
+                pleasant_pattern=0.0,
+                pleasant_perspective=0.0,
+                pleasant_post_processing=0.0,
+                pleasant_reflection=0.0,
+                pleasant_symmetry=0.0,
+                sharply_focused_subject=0.0,
+                tastefully_blurred=0.0,
+                well_chosen_subject=0.0,
+                well_framed_subject=0.0,
+                well_timed_shot=0.0,
+            )
+            return self._scoreinfo

osxphotos/photoinfo/photoinfo.py

@@ -5,28 +5,32 @@ PhotosDB.photos() returns a list of PhotoInfo objects
 """
 
 import dataclasses
-import glob
+import datetime
 import json
 import logging
 import os
 import os.path
 import pathlib
-import subprocess
-import sys
 from datetime import timedelta, timezone
-from pprint import pformat
 
 import yaml
 
 from .._constants import (
     _MOVIE_TYPE,
     _PHOTO_TYPE,
+    _PHOTOS_4_ALBUM_KIND,
+    _PHOTOS_4_ROOT_FOLDER,
     _PHOTOS_4_VERSION,
+    _PHOTOS_5_VERSION,
+    _PHOTOS_5_ALBUM_KIND,
+    _PHOTOS_5_IMPORT_SESSION_ALBUM_KIND,
+    _PHOTOS_5_SHARED_ALBUM_KIND,
     _PHOTOS_5_SHARED_PHOTO_PATH,
 )
-from ..albuminfo import AlbumInfo
+from ..albuminfo import AlbumInfo, ImportInfo
-from ..placeinfo import PlaceInfo4, PlaceInfo5
+from ..personinfo import FaceInfo, PersonInfo
 from ..phototemplate import PhotoTemplate
+from ..placeinfo import PlaceInfo4, PlaceInfo5
 from ..utils import _debug, _get_resource_loc, findfiles, get_preferred_uti_extension
 
 
@@ -55,6 +59,8 @@ class PhotoInfo:
         _xmp_sidecar,
         ExportResults,
     )
+    from ._photoinfo_scoreinfo import score, ScoreInfo
+    from ._photoinfo_comments import comments, likes 
 
     def __init__(self, db=None, uuid=None, info=None):
         self._uuid = uuid
@@ -64,10 +70,9 @@ class PhotoInfo:
     @property
     def filename(self):
         """ filename of the picture """
-        if self.has_raw and self.raw_original:
+        if self._db._db_version <= _PHOTOS_4_VERSION and self.has_raw and self.raw_original:
-            # return name of the RAW file
+            # return the JPEG version as that's what Photos 5+ does
-            # TODO: not yet implemented
+            return self._info["raw_pair_info"]["filename"]
-            return self._info["filename"]
         else:
             return self._info["filename"]
 
@@ -75,17 +80,16 @@ class PhotoInfo:
     def original_filename(self):
         """ original filename of the picture 
             Photos 5 mangles filenames upon import """
-        return self._info["originalFilename"]
+        if self._db._db_version <= _PHOTOS_4_VERSION and self.has_raw and self.raw_original:
+            # return the JPEG version as that's what Photos 5+ does
+            return self._info["raw_pair_info"]["originalFilename"]
+        else:
+            return self._info["originalFilename"]
 
     @property
     def date(self):
         """ image creation date as timezone aware datetime object """
-        imagedate = self._info["imageDate"]
+        return self._info["imageDate"]
-        seconds = self._info["imageTimeZoneOffsetSeconds"] or 0
-        delta = timedelta(seconds=seconds)
-        tz = timezone(delta)
-        imagedate_utc = imagedate.astimezone(tz=tz)
-        return imagedate_utc
 
     @property
     def date_modified(self):
@@ -96,8 +100,7 @@ class PhotoInfo:
             seconds = self._info["imageTimeZoneOffsetSeconds"] or 0
             delta = timedelta(seconds=seconds)
             tz = timezone(delta)
-            imagedate_utc = imagedate.astimezone(tz=tz)
+            return imagedate.astimezone(tz=tz)
-            return imagedate_utc
         else:
             return None
 
@@ -109,39 +112,67 @@ class PhotoInfo:
     @property
     def path(self):
         """ absolute path on disk of the original picture """
+        try:
+            return self._path
+        except AttributeError:
+            self._path = None
+            photopath = None
+            if self._info["isMissing"] == 1:
+                return photopath  # path would be meaningless until downloaded
 
-        photopath = None
+            if self._db._db_version <= _PHOTOS_4_VERSION:
-        if self._info["isMissing"] == 1:
+                if self._info["has_raw"]:
-            return photopath  # path would be meaningless until downloaded
+                    # return the path to JPEG even if RAW is original
+                    vol = (
+                        self._db._dbvolumes[self._info["raw_pair_info"]["volumeId"]]
+                        if self._info["raw_pair_info"]["volumeId"] is not None
+                        else None
+                    )
+                    if vol is not None:
+                        photopath = os.path.join(
+                            "/Volumes", vol, self._info["raw_pair_info"]["imagePath"]
+                        )
+                    else:
+                        photopath = os.path.join(
+                            self._db._masters_path,
+                            self._info["raw_pair_info"]["imagePath"],
+                        )
+                else:
+                    vol = self._info["volume"]
+                    if vol is not None:
+                        photopath = os.path.join(
+                            "/Volumes", vol, self._info["imagePath"]
+                        )
+                    else:
+                        photopath = os.path.join(
+                            self._db._masters_path, self._info["imagePath"]
+                        )
+                self._path = photopath
+                return photopath
 
-        if self._db._db_version <= _PHOTOS_4_VERSION:
+            if self._info["shared"]:
-            vol = self._info["volume"]
+                # shared photo
-            if vol is not None:
+                photopath = os.path.join(
-                photopath = os.path.join("/Volumes", vol, self._info["imagePath"])
+                    self._db._library_path,
+                    _PHOTOS_5_SHARED_PHOTO_PATH,
+                    self._info["directory"],
+                    self._info["filename"],
+                )
+                self._path = photopath
+                return photopath
+
+            if self._info["directory"].startswith("/"):
+                photopath = os.path.join(
+                    self._info["directory"], self._info["filename"]
+                )
             else:
                 photopath = os.path.join(
-                    self._db._masters_path, self._info["imagePath"]
+                    self._db._masters_path,
+                    self._info["directory"],
+                    self._info["filename"],
                 )
+            self._path = photopath
             return photopath
-            # TODO: Is there a way to use applescript or PhotoKit to force the download in this
-
-        if self._info["shared"]:
-            # shared photo
-            photopath = os.path.join(
-                self._db._library_path,
-                _PHOTOS_5_SHARED_PHOTO_PATH,
-                self._info["directory"],
-                self._info["filename"],
-            )
-            return photopath
-
-        if self._info["directory"].startswith("/"):
-            photopath = os.path.join(self._info["directory"], self._info["filename"])
-        else:
-            photopath = os.path.join(
-                self._db._masters_path, self._info["directory"], self._info["filename"]
-            )
-        return photopath
 
     @property
     def path_edited(self):
@@ -151,109 +182,132 @@ class PhotoInfo:
         # TODO: break this code into a _path_edited_4 and _path_edited_5
         # version to simplify the big if/then; same for path_live_photo
 
-        photopath = None
+        try:
-
+            return self._path_edited
-        if self._db._db_version <= _PHOTOS_4_VERSION:
+        except AttributeError:
-            if self._info["hasAdjustments"]:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
-                edit_id = self._info["edit_resource_id"]
+                self._path_edited = self._path_edited_4()
-                if edit_id is not None:
+                return self._path_edited
-                    library = self._db._library_path
-                    folder_id, file_id = _get_resource_loc(edit_id)
-                    # todo: is this always true or do we need to search file file_id under folder_id
-                    # figure out what kind it is and build filename
-                    filename = None
-                    if self._info["type"] == _PHOTO_TYPE:
-                        # it's a photo
-                        filename = f"fullsizeoutput_{file_id}.jpeg"
-                    elif self._info["type"] == _MOVIE_TYPE:
-                        # it's a movie
-                        filename = f"fullsizeoutput_{file_id}.mov"
-                    else:
-                        # don't know what it is!
-                        logging.debug(f"WARNING: unknown type {self._info['type']}")
-                        return None
-
-                    # photopath appears to usually be in "00" subfolder but
-                    # could be elsewhere--I haven't figured out this logic yet
-                    # first see if it's in 00
-                    photopath = os.path.join(
-                        library,
-                        "resources",
-                        "media",
-                        "version",
-                        folder_id,
-                        "00",
-                        filename,
-                    )
-
-                    if not os.path.isfile(photopath):
-                        rootdir = os.path.join(
-                            library, "resources", "media", "version", folder_id
-                        )
-
-                        for dirname, _, filelist in os.walk(rootdir):
-                            if filename in filelist:
-                                photopath = os.path.join(dirname, filename)
-                                break
-
-                    # check again to see if we found a valid file
-                    if not os.path.isfile(photopath):
-                        logging.debug(
-                            f"MISSING PATH: edited file for UUID {self._uuid} should be at {photopath} but does not appear to exist"
-                        )
-                        photopath = None
-                else:
-                    logging.debug(
-                        f"{self.uuid} hasAdjustments but edit_resource_id is None"
-                    )
-                    photopath = None
             else:
+                self._path_edited = self._path_edited_5()
+                return self._path_edited
+
+    def _path_edited_5(self):
+        """ return path_edited for Photos >= 5 """
+        # In Photos 5.0 / Catalina / MacOS 10.15:
+        # edited photos appear to always be converted to .jpeg and stored in
+        # library_name/resources/renders/X/UUID_1_201_a.jpeg
+        # where X = first letter of UUID
+        # and UUID = UUID of image
+        # this seems to be true even for photos not copied to Photos library and
+        # where original format was not jpg/jpeg
+        # if more than one edit, previous edit is stored as UUID_p.jpeg
+        #
+        # In Photos 6.0 / Big Sur, the edited image is a .heic if the photo isn't a jpeg,
+        # otherwise it's a jpeg.  It could also be a jpeg if photo library upgraded from earlier
+        # version.
+
+        if self._db._db_version < _PHOTOS_5_VERSION:
+            raise RuntimeError("Wrong database format!")
+
+        if self._info["hasAdjustments"]:
+            library = self._db._library_path
+            directory = self._uuid[0]  # first char of uuid
+            filename = None
+            if self._info["type"] == _PHOTO_TYPE:
+                # it's a photo
+                if self._db._photos_ver == 5:
+                    filename = f"{self._uuid}_1_201_a.jpeg"
+                else:
+                    # could be a heic or a jpeg
+                    if self.uti == "public.heic":
+                        filename = f"{self._uuid}_1_201_a.heic"
+                    else:
+                        filename = f"{self._uuid}_1_201_a.jpeg"
+            elif self._info["type"] == _MOVIE_TYPE:
+                # it's a movie
+                filename = f"{self._uuid}_2_0_a.mov"
+            else:
+                # don't know what it is!
+                logging.debug(f"WARNING: unknown type {self._info['type']}")
+                return None
+
+            photopath = os.path.join(
+                library, "resources", "renders", directory, filename
+            )
+
+            if not os.path.isfile(photopath):
+                logging.debug(
+                    f"edited file for UUID {self._uuid} should be at {photopath} but does not appear to exist"
+                )
                 photopath = None
-
-            # if self._info["isMissing"] == 1:
-            #     photopath = None  # path would be meaningless until downloaded
         else:
-            # in Photos 5.0 / Catalina / MacOS 10.15:
+            photopath = None
-            # edited photos appear to always be converted to .jpeg and stored in
-            # library_name/resources/renders/X/UUID_1_201_a.jpeg
-            # where X = first letter of UUID
-            # and UUID = UUID of image
-            # this seems to be true even for photos not copied to Photos library and
-            # where original format was not jpg/jpeg
-            # if more than one edit, previous edit is stored as UUID_p.jpeg
 
-            if self._info["hasAdjustments"]:
+        # TODO: might be possible for original/master to be missing but edit to still be there
+        # if self._info["isMissing"] == 1:
+        #     photopath = None  # path would be meaningless until downloaded
+
+        # logging.debug(photopath)
+
+        return photopath
+
+    def _path_edited_4(self):
+        """ return path_edited for Photos <= 4 """
+
+        if self._db._db_version > _PHOTOS_4_VERSION:
+            raise RuntimeError("Wrong database format!")
+
+        photopath = None
+        if self._info["hasAdjustments"]:
+            edit_id = self._info["edit_resource_id"]
+            if edit_id is not None:
                 library = self._db._library_path
-                directory = self._uuid[0]  # first char of uuid
+                folder_id, file_id = _get_resource_loc(edit_id)
+                # todo: is this always true or do we need to search file file_id under folder_id
+                # figure out what kind it is and build filename
                 filename = None
                 if self._info["type"] == _PHOTO_TYPE:
                     # it's a photo
-                    filename = f"{self._uuid}_1_201_a.jpeg"
+                    filename = f"fullsizeoutput_{file_id}.jpeg"
                 elif self._info["type"] == _MOVIE_TYPE:
                     # it's a movie
-                    filename = f"{self._uuid}_2_0_a.mov"
+                    filename = f"fullsizeoutput_{file_id}.mov"
                 else:
                     # don't know what it is!
                     logging.debug(f"WARNING: unknown type {self._info['type']}")
                     return None
 
+                # photopath appears to usually be in "00" subfolder but
+                # could be elsewhere--I haven't figured out this logic yet
+                # first see if it's in 00
                 photopath = os.path.join(
-                    library, "resources", "renders", directory, filename
+                    library, "resources", "media", "version", folder_id, "00", filename,
                 )
 
+                if not os.path.isfile(photopath):
+                    rootdir = os.path.join(
+                        library, "resources", "media", "version", folder_id
+                    )
+
+                    for dirname, _, filelist in os.walk(rootdir):
+                        if filename in filelist:
+                            photopath = os.path.join(dirname, filename)
+                            break
+
+                # check again to see if we found a valid file
                 if not os.path.isfile(photopath):
                     logging.debug(
-                        f"edited file for UUID {self._uuid} should be at {photopath} but does not appear to exist"
+                        f"MISSING PATH: edited file for UUID {self._uuid} should be at {photopath} but does not appear to exist"
                     )
                     photopath = None
             else:
+                logging.debug(
+                    f"{self.uuid} hasAdjustments but edit_resource_id is None"
+                )
                 photopath = None
-
+        else:
-            # TODO: might be possible for original/master to be missing but edit to still be there
+            photopath = None
-            # if self._info["isMissing"] == 1:
-            #     photopath = None  # path would be meaningless until downloaded
-
-            # logging.debug(photopath)
 
         return photopath
 
@@ -335,26 +389,69 @@ class PhotoInfo:
     @property
     def persons(self):
         """ list of persons in picture """
-        return self._info["persons"]
+        return [self._db._dbpersons_pk[pk]["fullname"] for pk in self._info["persons"]]
+
+    @property
+    def person_info(self):
+        """ list of PersonInfo objects for person in picture """
+        try:
+            return self._personinfo
+        except AttributeError:
+            self._personinfo = [
+                PersonInfo(db=self._db, pk=pk) for pk in self._info["persons"]
+            ]
+            return self._personinfo
+
+    @property
+    def face_info(self):
+        """ list of FaceInfo objects for faces in picture """
+        try:
+            return self._faceinfo
+        except AttributeError:
+            try:
+                faces = self._db._db_faceinfo_uuid[self._uuid]
+                self._faceinfo = [FaceInfo(db=self._db, pk=pk) for pk in faces]
+            except KeyError:
+                # no faces
+                self._faceinfo = []
+            return self._faceinfo
 
     @property
     def albums(self):
         """ list of albums picture is contained in """
-        albums = []
+        try:
-        for album in self._info["albums"]:
+            return self._albums
-            if not self._db._dbalbum_details[album]["intrash"]:
+        except AttributeError:
-                albums.append(self._db._dbalbum_details[album]["title"])
+            album_uuids = self._get_album_uuids()
-        return albums
+            self._albums = list(
+                {self._db._dbalbum_details[album]["title"] for album in album_uuids}
+            )
+            return self._albums
 
     @property
     def album_info(self):
         """ list of AlbumInfo objects representing albums the photos is contained in """
-        albums = []
+        try:
-        for album in self._info["albums"]:
+            return self._album_info
-            if not self._db._dbalbum_details[album]["intrash"]:
+        except AttributeError:
-                albums.append(AlbumInfo(db=self._db, uuid=album))
+            album_uuids = self._get_album_uuids()
+            self._album_info = [
+                AlbumInfo(db=self._db, uuid=album) for album in album_uuids
+            ]
+            return self._album_info
 
-        return albums
+    @property
+    def import_info(self):
+        """ ImportInfo object representing import session for the photo or None if no import session """
+        try:
+            return self._import_info
+        except AttributeError:
+            self._import_info = (
+                ImportInfo(db=self._db, uuid=self._info["import_uuid"])
+                if self._info["import_uuid"] is not None
+                else None
+            )
+            return self._import_info
 
     @property
     def keywords(self):
@@ -408,6 +505,11 @@ class PhotoInfo:
         """ True if picture is hidden """
         return True if self._info["hidden"] == 1 else False
 
+    @property
+    def intrash(self):
+        """ True if picture is in trash ('Recently Deleted' folder)"""
+        return self._info["intrash"]
+
     @property
     def location(self):
         """ returns (latitude, longitude) as float in degrees or None """
@@ -427,7 +529,40 @@ class PhotoInfo:
         """ Returns Uniform Type Identifier (UTI) for the image
             for example: public.jpeg or com.apple.quicktime-movie
         """
-        return self._info["UTI"]
+        if self._db._db_version <= _PHOTOS_4_VERSION:
+            if self.hasadjustments:
+                return self._info["UTI_edited"]
+            elif self.has_raw and self.raw_original:
+                # return UTI of the non-raw image to match Photos 5+ behavior
+                return self._info["raw_pair_info"]["UTI"]
+            else:
+                return self._info["UTI"]
+        else:
+            return self._info["UTI"]
+
+    @property
+    def uti_original(self):
+        """ Returns Uniform Type Identifier (UTI) for the original image
+            for example: public.jpeg or com.apple.quicktime-movie
+        """
+        if self._db._db_version <= _PHOTOS_4_VERSION and self._info["has_raw"]:
+            return self._info["raw_pair_info"]["UTI"]
+        elif self.shared:
+            # TODO: need reliable way to get original UTI for shared
+            return self.uti
+        else:
+            return self._info["UTI_original"]
+
+    @property
+    def uti_edited(self):
+        """ Returns Uniform Type Identifier (UTI) for the edited image 
+            if the photo has been edited, otherwise None; 
+            for example: public.jpeg 
+        """
+        if self._db._db_version >= _PHOTOS_5_VERSION:
+            return self.uti if self.hasadjustments else None
+        else:
+            return self._info["UTI_edited"]
 
     @property
     def uti_raw(self):
@@ -484,12 +619,11 @@ class PhotoInfo:
             self is not included in the returned list """
         if self._info["burst"]:
             burst_uuid = self._info["burstUUID"]
-            burst_photos = [
+            return [
                 PhotoInfo(db=self._db, uuid=u, info=self._db._dbphotos[u])
                 for u in self._db._dbphotos_burst[burst_uuid]
                 if u != self._uuid
             ]
-            return burst_photos
         else:
             return []
 
@@ -619,17 +753,67 @@ class PhotoInfo:
 
     @property
     def has_raw(self):
-        """ returns True if photo has an associated RAW image, otherwise False """
+        """ returns True if photo has an associated raw image (that is, it's a RAW+JPEG pair), otherwise False """
         return self._info["has_raw"]
 
+    @property
+    def israw(self):
+        """ returns True if photo is a raw image. For images with an associated RAW+JPEG pair, see has_raw """
+        return "raw-image" in self.uti_original
+
     @property
     def raw_original(self):
-        """ returns True if associated RAW image and the RAW image is selected in Photos
+        """ returns True if associated raw image and the raw image is selected in Photos
             via "Use RAW as Original "
             otherwise returns False """
         return self._info["raw_is_original"]
 
-    def render_template(self, template_str, none_str="_", path_sep=None):
+    @property
+    def height(self):
+        """ returns height of the current photo version in pixels """
+        return self._info["height"]
+
+    @property
+    def width(self):
+        """ returns width of the current photo version in pixels """
+        return self._info["width"]
+
+    @property
+    def orientation(self):
+        """ returns EXIF orientation of the current photo version as int """
+        return self._info["orientation"]
+
+    @property
+    def original_height(self):
+        """ returns height of the original photo version in pixels """
+        return self._info["original_height"]
+
+    @property
+    def original_width(self):
+        """ returns width of the original photo version in pixels """
+        return self._info["original_width"]
+
+    @property
+    def original_orientation(self):
+        """ returns EXIF orientation of the original photo version as int """
+        return self._info["original_orientation"]
+
+    @property
+    def original_filesize(self):
+        """ returns filesize of original photo in bytes as int """
+        return self._info["original_filesize"]
+
+    def render_template(
+        self,
+        template_str,
+        none_str="_",
+        path_sep=None,
+        expand_inplace=False,
+        inplace_sep=None,
+        filename=False,
+        dirname=False,
+        replacement=":",
+    ):
         """Renders a template string for PhotoInfo instance using PhotoTemplate
 
         Args:
@@ -637,12 +821,28 @@ class PhotoInfo:
             none_str: a str to use if template field renders to None, default is "_".
             path_sep: a single character str to use as path separator when joining 
                 fields like folder_album; if not provided, defaults to os.path.sep
+            expand_inplace: expand multi-valued substitutions in-place as a single string 
+                instead of returning individual strings
+            inplace_sep: optional string to use as separator between multi-valued keywords
+                with expand_inplace; default is ','
+            filename: if True, template output will be sanitized to produce valid file name
+            dirname: if True, template output will be sanitized to produce valid directory name 
+            replacement: str, value to replace any illegal file path characters with; default = ":"
         
         Returns:
             ([rendered_strings], [unmatched]): tuple of list of rendered strings and list of unmatched template values
         """
         template = PhotoTemplate(self)
-        return template.render(template_str, none_str, path_sep)
+        return template.render(
+            template_str,
+            none_str=none_str,
+            path_sep=path_sep,
+            expand_inplace=expand_inplace,
+            inplace_sep=inplace_sep,
+            filename=filename,
+            dirname=dirname,
+            replacement=replacement
+        )
 
     @property
     def _longitude(self):
@@ -654,6 +854,37 @@ class PhotoInfo:
         """ Returns latitude, in degrees """
         return self._info["latitude"]
 
+    def _get_album_uuids(self):
+        """ Return list of album UUIDs this photo is found in
+        
+            Filters out albums in the trash and any special album types
+
+        Returns: list of album UUIDs 
+        """
+        if self._db._db_version <= _PHOTOS_4_VERSION:
+            version4 = True
+            album_kind = [_PHOTOS_4_ALBUM_KIND]
+        else:
+            version4 = False
+            album_kind = [_PHOTOS_5_SHARED_ALBUM_KIND, _PHOTOS_5_ALBUM_KIND]
+
+        album_list = []
+        for album in self._info["albums"]:
+            detail = self._db._dbalbum_details[album]
+            if (
+                detail["kind"] in album_kind
+                and not detail["intrash"]
+                and (
+                    not version4
+                    # in Photos <= 4, special albums like "printAlbum" have kind _PHOTOS_4_ALBUM_KIND
+                    # but should not be listed here; they can be distinguished by looking
+                    # for folderUuid of _PHOTOS_4_ROOT_FOLDER as opposed to _PHOTOS_4_TOP_LEVEL_ALBUM
+                    or (version4 and detail["folderUuid"] != _PHOTOS_4_ROOT_FOLDER)
+                )
+            ):
+                album_list.append(album)
+        return album_list
+
     def __repr__(self):
         return f"osxphotos.{self.__class__.__name__}(db={self._db}, uuid='{self._uuid}', info={self._info})"
 
@@ -664,6 +895,8 @@ class PhotoInfo:
         date_modified_iso = (
             self.date_modified.isoformat() if self.date_modified else None
         )
+        exif = str(self.exif_info) if self.exif_info else None
+        score = str(self.score) if self.score else None
 
         info = {
             "uuid": self.uuid,
@@ -704,24 +937,37 @@ class PhotoInfo:
             "has_raw": self.has_raw,
             "uti_raw": self.uti_raw,
             "path_raw": self.path_raw,
+            "place": self.place,
+            "exif": exif,
+            "score": score,
+            "intrash": self.intrash,
+            "height": self.height,
+            "width": self.width,
+            "orientation": self.orientation,
+            "original_height": self.original_height,
+            "original_width": self.original_width,
+            "original_orientation": self.original_orientation,
+            "original_filesize": self.original_filesize,
         }
         return yaml.dump(info, sort_keys=False)
 
-    def json(self):
+    def asdict(self):
-        """ return JSON representation """
+        """ return dict representation """
 
-        date_modified_iso = (
-            self.date_modified.isoformat() if self.date_modified else None
-        )
         folders = {album.title: album.folder_names for album in self.album_info}
         exif = dataclasses.asdict(self.exif_info) if self.exif_info else {}
-        place = self.place.as_dict() if self.place else {}
+        place = self.place.asdict() if self.place else {}
+        score = dataclasses.asdict(self.score) if self.score else {}
+        comments = [comment.asdict() for comment in self.comments]
+        likes = [like.asdict() for like in self.likes]
+        faces = [face.asdict() for face in self.face_info]
 
-        pic = {
+        return {
+            "library": self._db._library_path,
             "uuid": self.uuid,
             "filename": self.filename,
             "original_filename": self.original_filename,
-            "date": self.date.isoformat(),
+            "date": self.date,
             "description": self.description,
             "title": self.title,
             "keywords": self.keywords,
@@ -730,6 +976,7 @@ class PhotoInfo:
             "albums": self.albums,
             "folders": folders,
             "persons": self.persons,
+            "faces": faces,
             "path": self.path,
             "ismissing": self.ismissing,
             "hasadjustments": self.hasadjustments,
@@ -743,12 +990,13 @@ class PhotoInfo:
             "isphoto": self.isphoto,
             "ismovie": self.ismovie,
             "uti": self.uti,
+            "uti_original": self.uti_original,
             "burst": self.burst,
             "live_photo": self.live_photo,
             "path_live_photo": self.path_live_photo,
             "iscloudasset": self.iscloudasset,
             "incloud": self.incloud,
-            "date_modified": date_modified_iso,
+            "date_modified": self.date_modified,
             "portrait": self.portrait,
             "screenshot": self.screenshot,
             "slow_mo": self.slow_mo,
@@ -757,19 +1005,45 @@ class PhotoInfo:
             "selfie": self.selfie,
             "panorama": self.panorama,
             "has_raw": self.has_raw,
+            "israw": self.israw,
+            "raw_original": self.raw_original,
             "uti_raw": self.uti_raw,
             "path_raw": self.path_raw,
             "place": place,
             "exif": exif,
+            "score": score,
+            "intrash": self.intrash,
+            "height": self.height,
+            "width": self.width,
+            "orientation": self.orientation,
+            "original_height": self.original_height,
+            "original_width": self.original_width,
+            "original_orientation": self.original_orientation,
+            "original_filesize": self.original_filesize,
+            "comments": comments,
+            "likes": likes,
         }
-        return json.dumps(pic)
 
-    # compare two PhotoInfo objects for equality
+    def json(self):
+        """ Return JSON representation """
+        def default(o):
+            if isinstance(o, (datetime.date, datetime.datetime)):
+                return o.isoformat()
+
+        return json.dumps(self.asdict(), sort_keys=True, default=default)
+
     def __eq__(self, other):
+        """ Compare two PhotoInfo objects for equality """
+        # Can't just compare the two __dicts__ because some methods (like albums)
+        # memoize their value once called in an instance variable (e.g. self._albums)
         if isinstance(other, self.__class__):
-            return self.__dict__ == other.__dict__
+            return (
-
+                self._db.db_path == other._db.db_path
+                and self.uuid == other.uuid
+                and self._info == other._info
+            )
         return False
 
     def __ne__(self, other):
+        """ Compare two PhotoInfo objects for inequality """
         return not self.__eq__(other)

osxphotos/photosdb/__init__.py

@@ -4,3 +4,4 @@ Processes a Photos.app library database to extract information about photos
 """
 
 from .photosdb import PhotosDB
+from .photosdb_utils import get_db_version, get_db_model_version, get_model_version

osxphotos/photosdb/_photosdb_process_comments.py

@@ -0,0 +1,157 @@
+""" PhotosDB method for processing comments and likes on shared photos.
+    Do not import this module directly """
+
+import dataclasses
+import datetime
+from dataclasses import dataclass
+
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION, TIME_DELTA
+from ..utils import _open_sql_file, normalize_unicode
+
+
+def _process_comments(self):
+    """ load the comments and likes data from the database 
+        this is a PhotosDB method that should be imported in 
+        the PhotosDB class definition in photosdb.py
+    """
+    self._db_hashed_person_id = {}
+    self._db_comments_uuid = {}
+    if self._db_version <= _PHOTOS_4_VERSION:
+        _process_comments_4(self)
+    else:
+        _process_comments_5(self)
+
+
+@dataclass
+class CommentInfo:
+    """ Class for shared photo comments """
+
+    datetime: datetime.datetime
+    user: str
+    ismine: bool
+    text: str
+
+    def asdict(self):
+        return dataclasses.asdict(self)
+
+
+@dataclass
+class LikeInfo:
+    """ Class for shared photo likes """
+
+    datetime: datetime.datetime
+    user: str
+    ismine: bool
+
+    def asdict(self):
+        return dataclasses.asdict(self)
+
+
+# The following methods do not get imported into PhotosDB
+# but will get called by _process_comments
+def _process_comments_4(photosdb):
+    """ process comments and likes info for Photos <= 4 
+        photosdb: PhotosDB instance """
+    raise NotImplementedError(
+        f"Not implemented for database version {photosdb._db_version}."
+    )
+
+
+def _process_comments_5(photosdb):
+    """ process comments and likes info for Photos >= 5 
+        photosdb: PhotosDB instance """
+
+    db = photosdb._tmp_db
+
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+
+    (conn, cursor) = _open_sql_file(db)
+
+    results = conn.execute(
+        """
+        SELECT DISTINCT
+        ZINVITEEHASHEDPERSONID,
+        ZINVITEEFIRSTNAME,
+        ZINVITEELASTNAME,
+        ZINVITEEFULLNAME
+        FROM
+        ZCLOUDSHAREDALBUMINVITATIONRECORD
+        """
+    )
+
+    # order of results
+    # 0: ZINVITEEHASHEDPERSONID,
+    # 1: ZINVITEEFIRSTNAME,
+    # 2: ZINVITEELASTNAME,
+    # 3: ZINVITEEFULLNAME
+
+    photosdb._db_hashed_person_id = {}
+    for row in results.fetchall():
+        person_id = row[0]
+        photosdb._db_hashed_person_id[person_id] = {
+            "first_name": normalize_unicode(row[1]),
+            "last_name": normalize_unicode(row[2]),
+            "full_name": normalize_unicode(row[3]),
+        }
+
+    results = conn.execute(
+        f"""
+        SELECT 
+        {asset_table}.ZUUID, -- UUID of the photo
+        ZCLOUDSHAREDCOMMENT.ZISLIKE, -- comment is actually a "like"
+        ZCLOUDSHAREDCOMMENT.ZCOMMENTDATE, -- date of comment
+        ZCLOUDSHAREDCOMMENT.ZCOMMENTTEXT, -- text of comment
+        ZCLOUDSHAREDCOMMENT.ZCOMMENTERHASHEDPERSONID, -- hashed ID of person who made comment/like
+        ZCLOUDSHAREDCOMMENT.ZISMYCOMMENT -- is my (this user's) comment
+        FROM ZCLOUDSHAREDCOMMENT
+        JOIN {asset_table} ON
+        {asset_table}.Z_PK = ZCLOUDSHAREDCOMMENT.ZCOMMENTEDASSET
+        OR
+        {asset_table}.Z_PK = ZCLOUDSHAREDCOMMENT.ZLIKEDASSET
+    """
+    )
+
+    # order of results
+    # 0: ZGENERICASSET.ZUUID, -- UUID of the photo
+    # 1: ZCLOUDSHAREDCOMMENT.ZISLIKE, -- comment is actually a "like"
+    # 2: ZCLOUDSHAREDCOMMENT.ZCOMMENTDATE, -- date of comment
+    # 3: ZCLOUDSHAREDCOMMENT.ZCOMMENTTEXT, -- text of comment
+    # 4: ZCLOUDSHAREDCOMMENT.ZCOMMENTERHASHEDPERSONID, -- hashed ID of person who made comment/like
+    # 5: ZCLOUDSHAREDCOMMENT.ZISMYCOMMENT -- is my (this user's) comment
+
+    photosdb._db_comments_uuid = {}
+    for row in results:
+        uuid = row[0]
+        is_like = bool(row[1])
+        text = normalize_unicode(row[3])
+        try:
+            user_name = photosdb._db_hashed_person_id[row[4]]["full_name"]
+        except KeyError:
+            user_name = None
+
+        try:
+            dt = datetime.datetime.fromtimestamp(row[2] + TIME_DELTA)
+        except:
+            dt = datetime.datetime(1970, 1, 1)
+
+        ismine = bool(row[5])
+
+        try:
+            db_comments = photosdb._db_comments_uuid[uuid]
+        except KeyError:
+            photosdb._db_comments_uuid[uuid] = {"likes": [], "comments": []}
+            db_comments = photosdb._db_comments_uuid[uuid]
+
+        if is_like:
+            db_comments["likes"].append(LikeInfo(dt, user_name, ismine))
+        elif text:
+            db_comments["comments"].append(CommentInfo(dt, user_name, ismine, text))
+
+    # sort results
+    for uuid in photosdb._db_comments_uuid:
+        if photosdb._db_comments_uuid[uuid]["likes"]:
+            photosdb._db_comments_uuid[uuid]["likes"].sort(key=lambda x: x.datetime)
+        if photosdb._db_comments_uuid[uuid]["comments"]:
+            photosdb._db_comments_uuid[uuid]["comments"].sort(key=lambda x: x.datetime)
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_exif.py

@@ -3,9 +3,9 @@
 
 import logging
 
-from .._constants import _PHOTOS_4_VERSION
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION
 from ..utils import _db_is_locked, _debug, _open_sql_file
-
+from .photosdb_utils import get_db_version
 
 def _process_exifinfo(self):
     """ load the exif data from the database 
@@ -35,14 +35,16 @@ def _process_exifinfo_5(photosdb):
 
     db = photosdb._tmp_db
 
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+        
     (conn, cursor) = _open_sql_file(db)
 
     result = conn.execute(
-        """
+        f"""
-        SELECT ZGENERICASSET.ZUUID, ZEXTENDEDATTRIBUTES.*
+        SELECT {asset_table}.ZUUID, ZEXTENDEDATTRIBUTES.*
-        FROM ZGENERICASSET
+        FROM {asset_table}
         JOIN ZEXTENDEDATTRIBUTES
-        ON ZEXTENDEDATTRIBUTES.ZASSET = ZGENERICASSET.Z_PK
+        ON ZEXTENDEDATTRIBUTES.ZASSET = {asset_table}.Z_PK
         """
     )
 
@@ -54,3 +56,5 @@ def _process_exifinfo_5(photosdb):
         if uuid in photosdb._db_exifinfo_uuid:
             logging.warning(f"duplicate exifinfo record found for uuid {uuid}")
         photosdb._db_exifinfo_uuid[uuid] = record
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_faceinfo.py

@@ -0,0 +1,331 @@
+""" Methods for PhotosDB to add Photos face info 
+"""
+
+import logging
+
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION
+from ..utils import _open_sql_file, normalize_unicode
+from .photosdb_utils import get_db_version
+
+
+"""
+    This module should be imported in the class defintion of PhotosDB in photosdb.py
+    Do not import this module directly
+    This module adds the following method to PhotosDB:
+        _process_faceinfo: process photo face info 
+
+    The following data structures are added to PhotosDB
+        self._db_faceinfo_pk: {pk: {faceinfo}}
+        self._db_faceinfo_uuid: {photo uuid: [face pk]}
+        self._db_faceinfo_person: {person_pk: [face_pk]}
+"""
+
+
+def _process_faceinfo(self):
+    """ Process face information
+    """
+
+    self._db_faceinfo_pk = {}
+    self._db_faceinfo_uuid = {}
+    self._db_faceinfo_person = {}
+
+    if self._db_version <= _PHOTOS_4_VERSION:
+        _process_faceinfo_4(self)
+    else:
+        _process_faceinfo_5(self)
+
+
+def _process_faceinfo_4(photosdb):
+    """ Process face information for Photos 4 databases
+
+    Args:
+        photosdb: an OSXPhotosDB instance
+    """
+    db = photosdb._tmp_db
+
+    (conn, cursor) = _open_sql_file(db)
+
+    result = cursor.execute(
+        """
+        SELECT
+        RKFace.modelId,
+        RKVersion.uuid, 
+        RKFace.uuid,
+        RKPerson.name,
+        RKFace.isInTrash,
+        RKFace.personId,
+        RKFace.imageModelId, 
+        RKFace.sourceWidth,
+        RKFace.sourceHeight, 
+        RKFace.centerX,
+        RKFace.centerY,
+        RKFace.size,
+        RKFace.leftEyeX, 
+        RKFace.leftEyeY,
+        RKFace.rightEyeX, 
+        RKFace.rightEyeY, 
+        RKFace.mouthX,
+        RKFace.mouthY, 
+        RKFace.hidden, 
+        RKFace.manual, 
+        RKFace.hasSmile, 
+        RKFace.isLeftEyeClosed, 
+        RKFace.isRightEyeClosed, 
+        RKFace.poseRoll, 
+        RKFace.poseYaw, 
+        RKFace.posePitch, 
+        RKFace.faceType,
+        RKFace.qualityMeasure
+        FROM
+        RKFace
+        JOIN RKPerson on RKPerson.modelId = RKFace.personId
+        JOIN RKVersion on RKVersion.modelId = RKFace.imageModelId
+        """
+    )
+
+    # 0     RKFace.modelId,
+    # 1     RKVersion.uuid,
+    # 2     RKFace.uuid,
+    # 3     RKPerson.name,
+    # 4     RKFace.isInTrash,
+    # 5     RKFace.personId,
+    # 6     RKFace.imageModelId,
+    # 7     RKFace.sourceWidth,
+    # 8     RKFace.sourceHeight,
+    # 9     RKFace.centerX,
+    # 10    RKFace.centerY,
+    # 11    RKFace.size,
+    # 12    RKFace.leftEyeX,
+    # 13    RKFace.leftEyeY,
+    # 14    RKFace.rightEyeX,
+    # 15    RKFace.rightEyeY,
+    # 16    RKFace.mouthX,
+    # 17    RKFace.mouthY,
+    # 18    RKFace.hidden,
+    # 19    RKFace.manual,
+    # 20    RKFace.hasSmile,
+    # 21    RKFace.isLeftEyeClosed,
+    # 22    RKFace.isRightEyeClosed,
+    # 23    RKFace.poseRoll,
+    # 24    RKFace.poseYaw,
+    # 25    RKFace.posePitch,
+    # 26    RKFace.faceType,
+    # 27    RKFace.qualityMeasure
+
+    for row in result:
+        modelid = row[0]
+        asset_uuid = row[1]
+        person_id = row[5]
+        face = {}
+        face["pk"] = modelid
+        face["asset_uuid"] = asset_uuid
+        face["uuid"] = row[2]
+        face["person"] = person_id
+        face["fullname"] = normalize_unicode(row[3])
+        face["sourcewidth"] = row[7]
+        face["sourceheight"] = row[8]
+        face["centerx"] = row[9]
+        face["centery"] = row[10]
+        face["size"] = row[11]
+        face["lefteyex"] = row[12]
+        face["lefteyey"] = row[13]
+        face["righteyex"] = row[14]
+        face["righteyey"] = row[15]
+        face["mouthx"] = row[16]
+        face["mouthy"] = row[17]
+        face["hidden"] = row[18]
+        face["manual"] = row[19]
+        face["has_smile"] = row[20]
+        face["left_eye_closed"] = row[21]
+        face["right_eye_closed"] = row[22]
+        face["roll"] = row[23]
+        face["yaw"] = row[24]
+        face["pitch"] = row[25]
+        face["facetype"] = row[26]
+        face["quality"] = row[27]
+
+        # Photos 5 only
+        face["agetype"] = None
+        face["baldtype"] = None
+        face["eyemakeuptype"] = None
+        face["eyestate"] = None
+        face["facialhairtype"] = None
+        face["gendertype"] = None
+        face["glassestype"] = None
+        face["haircolortype"] = None
+        face["intrash"] = None
+        face["lipmakeuptype"] = None
+        face["smiletype"] = None
+
+        photosdb._db_faceinfo_pk[modelid] = face
+
+        try:
+            photosdb._db_faceinfo_uuid[asset_uuid].append(modelid)
+        except KeyError:
+            photosdb._db_faceinfo_uuid[asset_uuid] = [modelid]
+
+        try:
+            photosdb._db_faceinfo_person[person_id].append(modelid)
+        except KeyError:
+            photosdb._db_faceinfo_person[person_id] = [modelid]
+
+    conn.close()
+
+
+def _process_faceinfo_5(photosdb):
+    """ Process face information for Photos 5 databases
+
+    Args:
+        photosdb: an OSXPhotosDB instance
+    """
+
+    db = photosdb._tmp_db
+
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+
+    (conn, cursor) = _open_sql_file(db)
+
+    result = cursor.execute(
+        f""" 
+        SELECT
+        ZDETECTEDFACE.Z_PK,
+        {asset_table}.ZUUID,
+        ZDETECTEDFACE.ZUUID,
+        ZDETECTEDFACE.ZPERSON,
+        ZPERSON.ZFULLNAME,
+        ZDETECTEDFACE.ZAGETYPE,
+        ZDETECTEDFACE.ZBALDTYPE,
+        ZDETECTEDFACE.ZEYEMAKEUPTYPE,
+        ZDETECTEDFACE.ZEYESSTATE,
+        ZDETECTEDFACE.ZFACIALHAIRTYPE,
+        ZDETECTEDFACE.ZGENDERTYPE,
+        ZDETECTEDFACE.ZGLASSESTYPE,
+        ZDETECTEDFACE.ZHAIRCOLORTYPE,
+        ZDETECTEDFACE.ZHASSMILE,
+        ZDETECTEDFACE.ZHIDDEN,
+        ZDETECTEDFACE.ZISINTRASH,
+        ZDETECTEDFACE.ZISLEFTEYECLOSED,
+        ZDETECTEDFACE.ZISRIGHTEYECLOSED,
+        ZDETECTEDFACE.ZLIPMAKEUPTYPE,
+        ZDETECTEDFACE.ZMANUAL,
+        ZDETECTEDFACE.ZQUALITYMEASURE,
+        ZDETECTEDFACE.ZSMILETYPE,
+        ZDETECTEDFACE.ZSOURCEHEIGHT,
+        ZDETECTEDFACE.ZSOURCEWIDTH,
+        ZDETECTEDFACE.ZBLURSCORE,
+        ZDETECTEDFACE.ZCENTERX,
+        ZDETECTEDFACE.ZCENTERY,
+        ZDETECTEDFACE.ZLEFTEYEX,
+        ZDETECTEDFACE.ZLEFTEYEY,
+        ZDETECTEDFACE.ZMOUTHX,
+        ZDETECTEDFACE.ZMOUTHY,
+        ZDETECTEDFACE.ZPOSEYAW,
+        ZDETECTEDFACE.ZQUALITY,
+        ZDETECTEDFACE.ZRIGHTEYEX,
+        ZDETECTEDFACE.ZRIGHTEYEY,
+        ZDETECTEDFACE.ZROLL,
+        ZDETECTEDFACE.ZSIZE,
+        ZDETECTEDFACE.ZYAW,
+        ZDETECTEDFACE.ZMASTERIDENTIFIER
+        FROM ZDETECTEDFACE
+        JOIN {asset_table} ON {asset_table}.Z_PK = ZDETECTEDFACE.ZASSET
+        JOIN ZPERSON ON ZPERSON.Z_PK = ZDETECTEDFACE.ZPERSON;
+        """
+    )
+
+    # 0    ZDETECTEDFACE.Z_PK
+    # 1    ZGENERICASSET.ZUUID,
+    # 2    ZDETECTEDFACE.ZUUID,
+    # 3    ZDETECTEDFACE.ZPERSON,
+    # 4    ZPERSON.ZFULLNAME,
+    # 5    ZDETECTEDFACE.ZAGETYPE,
+    # 6    ZDETECTEDFACE.ZBALDTYPE,
+    # 7    ZDETECTEDFACE.ZEYEMAKEUPTYPE,
+    # 8    ZDETECTEDFACE.ZEYESSTATE,
+    # 9    ZDETECTEDFACE.ZFACIALHAIRTYPE,
+    # 10   ZDETECTEDFACE.ZGENDERTYPE,
+    # 11   ZDETECTEDFACE.ZGLASSESTYPE,
+    # 12   ZDETECTEDFACE.ZHAIRCOLORTYPE,
+    # 13   ZDETECTEDFACE.ZHASSMILE,
+    # 14   ZDETECTEDFACE.ZHIDDEN,
+    # 15   ZDETECTEDFACE.ZISINTRASH,
+    # 16   ZDETECTEDFACE.ZISLEFTEYECLOSED,
+    # 17   ZDETECTEDFACE.ZISRIGHTEYECLOSED,
+    # 18   ZDETECTEDFACE.ZLIPMAKEUPTYPE,
+    # 19   ZDETECTEDFACE.ZMANUAL,
+    # 20   ZDETECTEDFACE.ZQUALITYMEASURE,
+    # 21   ZDETECTEDFACE.ZSMILETYPE,
+    # 22   ZDETECTEDFACE.ZSOURCEHEIGHT,
+    # 23   ZDETECTEDFACE.ZSOURCEWIDTH,
+    # 24   ZDETECTEDFACE.ZBLURSCORE,
+    # 25   ZDETECTEDFACE.ZCENTERX,
+    # 26   ZDETECTEDFACE.ZCENTERY,
+    # 27   ZDETECTEDFACE.ZLEFTEYEX,
+    # 28   ZDETECTEDFACE.ZLEFTEYEY,
+    # 29   ZDETECTEDFACE.ZMOUTHX,
+    # 30   ZDETECTEDFACE.ZMOUTHY,
+    # 31   ZDETECTEDFACE.ZPOSEYAW,
+    # 32   ZDETECTEDFACE.ZQUALITY,
+    # 33   ZDETECTEDFACE.ZRIGHTEYEX,
+    # 34   ZDETECTEDFACE.ZRIGHTEYEY,
+    # 35   ZDETECTEDFACE.ZROLL,
+    # 36   ZDETECTEDFACE.ZSIZE,
+    # 37   ZDETECTEDFACE.ZYAW,
+    # 38   ZDETECTEDFACE.ZMASTERIDENTIFIER
+
+    for row in result:
+        pk = row[0]
+        asset_uuid = row[1]
+        person_pk = row[3]
+        face = {}
+        face["pk"] = pk
+        face["asset_uuid"] = asset_uuid
+        face["uuid"] = row[2]
+        face["person"] = person_pk
+        face["fullname"] = normalize_unicode(row[4])
+        face["agetype"] = row[5]
+        face["baldtype"] = row[6]
+        face["eyemakeuptype"] = row[7]
+        face["eyestate"] = row[8]
+        face["facialhairtype"] = row[9]
+        face["gendertype"] = row[10]
+        face["glassestype"] = row[11]
+        face["haircolortype"] = row[12]
+        face["has_smile"] = row[13]
+        face["hidden"] = row[14]
+        face["intrash"] = row[15]
+        face["left_eye_closed"] = row[16]
+        face["right_eye_closed"] = row[17]
+        face["lipmakeuptype"] = row[18]
+        face["manual"] = row[19]
+        face["smiletype"] = row[21]
+        face["sourceheight"] = row[22]
+        face["sourcewidth"] = row[23]
+        face["facetype"] = None  # Photos 4 only
+        face["centerx"] = row[25]
+        face["centery"] = row[26]
+        face["lefteyex"] = row[27]
+        face["lefteyey"] = row[28]
+        face["mouthx"] = row[29]
+        face["mouthy"] = row[30]
+        face["quality"] = row[32]
+        face["righteyex"] = row[33]
+        face["righteyey"] = row[34]
+        face["roll"] = row[35]
+        face["size"] = row[36]
+        face["yaw"] = row[37]
+        face["pitch"] = 0.0  # not defined in Photos 5
+
+        photosdb._db_faceinfo_pk[pk] = face
+
+        try:
+            photosdb._db_faceinfo_uuid[asset_uuid].append(pk)
+        except KeyError:
+            photosdb._db_faceinfo_uuid[asset_uuid] = [pk]
+
+        try:
+            photosdb._db_faceinfo_person[person_pk].append(pk)
+        except KeyError:
+            photosdb._db_faceinfo_person[person_pk] = [pk]
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_scoreinfo.py

@@ -0,0 +1,150 @@
+""" Methods for PhotosDB to add Photos 5 photo score info 
+    ref: https://simonwillison.net/2020/May/21/dogsheep-photos/
+"""
+
+import logging
+
+from .._constants import _DB_TABLE_NAMES, _PHOTOS_4_VERSION
+from ..utils import _open_sql_file
+from .photosdb_utils import get_db_version
+
+"""
+    This module should be imported in the class defintion of PhotosDB in photosdb.py
+    Do not import this module directly
+    This module adds the following method to PhotosDB:
+        _process_scoreinfo: process photo score info 
+
+    The following data structures are added to PhotosDB
+        self._db_scoreinfo_uuid
+    
+    These methods only work on Photos 5 databases.  Will print warning on earlier library versions.
+"""
+
+
+def _process_scoreinfo(self):
+    """ Process computed photo scores
+        Note: Only works on Photos version == 5.0
+    """
+
+    # _db_scoreinfo_uuid is dict in form {uuid: {score values}}
+    self._db_scoreinfo_uuid = {}
+
+    if self._db_version <= _PHOTOS_4_VERSION:
+        raise NotImplementedError(
+            f"search info not implemented for this database version"
+        )
+    else:
+        _process_scoreinfo_5(self)
+
+
+def _process_scoreinfo_5(photosdb):
+    """ Process computed photo scores for Photos 5 databases
+
+    Args:
+        photosdb: an OSXPhotosDB instance
+    """
+
+    db = photosdb._tmp_db
+
+    asset_table = _DB_TABLE_NAMES[photosdb._photos_ver]["ASSET"]
+
+    (conn, cursor) = _open_sql_file(db)
+
+    result = cursor.execute(
+        f"""
+        SELECT 
+        {asset_table}.ZUUID,
+        {asset_table}.ZOVERALLAESTHETICSCORE,
+        {asset_table}.ZCURATIONSCORE,
+        {asset_table}.ZPROMOTIONSCORE,
+        {asset_table}.ZHIGHLIGHTVISIBILITYSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZBEHAVIORALSCORE, 
+        ZCOMPUTEDASSETATTRIBUTES.ZFAILURESCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZHARMONIOUSCOLORSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZIMMERSIVENESSSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZINTERACTIONSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZINTERESTINGSUBJECTSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZINTRUSIVEOBJECTPRESENCESCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZLIVELYCOLORSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZLOWLIGHT,
+        ZCOMPUTEDASSETATTRIBUTES.ZNOISESCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTCAMERATILTSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTCOMPOSITIONSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTLIGHTINGSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPATTERNSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPERSPECTIVESCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPOSTPROCESSINGSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTREFLECTIONSSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTSYMMETRYSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZSHARPLYFOCUSEDSUBJECTSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZTASTEFULLYBLURREDSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZWELLCHOSENSUBJECTSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZWELLFRAMEDSUBJECTSCORE,
+        ZCOMPUTEDASSETATTRIBUTES.ZWELLTIMEDSHOTSCORE
+        FROM {asset_table}
+        JOIN ZCOMPUTEDASSETATTRIBUTES ON ZCOMPUTEDASSETATTRIBUTES.ZASSET = {asset_table}.Z_PK
+        """
+    )
+
+    # 0     ZGENERICASSET.ZUUID,
+    # 1     ZGENERICASSET.ZOVERALLAESTHETICSCORE,
+    # 2     ZGENERICASSET.ZCURATIONSCORE,
+    # 3     ZGENERICASSET.ZPROMOTIONSCORE,
+    # 4     ZGENERICASSET.ZHIGHLIGHTVISIBILITYSCORE,
+    # 5     ZCOMPUTEDASSETATTRIBUTES.ZBEHAVIORALSCORE,
+    # 6     ZCOMPUTEDASSETATTRIBUTES.ZFAILURESCORE,
+    # 7     ZCOMPUTEDASSETATTRIBUTES.ZHARMONIOUSCOLORSCORE,
+    # 8     ZCOMPUTEDASSETATTRIBUTES.ZIMMERSIVENESSSCORE,
+    # 9     ZCOMPUTEDASSETATTRIBUTES.ZINTERACTIONSCORE,
+    # 10    ZCOMPUTEDASSETATTRIBUTES.ZINTERESTINGSUBJECTSCORE,
+    # 11    ZCOMPUTEDASSETATTRIBUTES.ZINTRUSIVEOBJECTPRESENCESCORE,
+    # 12    ZCOMPUTEDASSETATTRIBUTES.ZLIVELYCOLORSCORE,
+    # 13    ZCOMPUTEDASSETATTRIBUTES.ZLOWLIGHT,
+    # 14    ZCOMPUTEDASSETATTRIBUTES.ZNOISESCORE,
+    # 15    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTCAMERATILTSCORE,
+    # 16    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTCOMPOSITIONSCORE,
+    # 17    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTLIGHTINGSCORE,
+    # 18    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPATTERNSCORE,
+    # 19    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPERSPECTIVESCORE,
+    # 20    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPOSTPROCESSINGSCORE,
+    # 21    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTREFLECTIONSSCORE,
+    # 22    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTSYMMETRYSCORE,
+    # 23    ZCOMPUTEDASSETATTRIBUTES.ZSHARPLYFOCUSEDSUBJECTSCORE,
+    # 24    ZCOMPUTEDASSETATTRIBUTES.ZTASTEFULLYBLURREDSCORE,
+    # 25    ZCOMPUTEDASSETATTRIBUTES.ZWELLCHOSENSUBJECTSCORE,
+    # 26    ZCOMPUTEDASSETATTRIBUTES.ZWELLFRAMEDSUBJECTSCORE,
+    # 27    ZCOMPUTEDASSETATTRIBUTES.ZWELLTIMEDSHOTSCORE
+
+    for row in result:
+        uuid = row[0]
+        scores = {"uuid": uuid}
+        scores["overall_aesthetic"] = row[1]
+        scores["curation"] = row[2]
+        scores["promotion"] = row[3]
+        scores["highlight_visibility"] = row[4]
+        scores["behavioral"] = row[5]
+        scores["failure"] = row[6]
+        scores["harmonious_color"] = row[7]
+        scores["immersiveness"] = row[8]
+        scores["interaction"] = row[9]
+        scores["interesting_subject"] = row[10]
+        scores["intrusive_object_presence"] = row[11]
+        scores["lively_color"] = row[12]
+        scores["low_light"] = row[13]
+        scores["noise"] = row[14]
+        scores["pleasant_camera_tilt"] = row[15]
+        scores["pleasant_composition"] = row[16]
+        scores["pleasant_lighting"] = row[17]
+        scores["pleasant_pattern"] = row[18]
+        scores["pleasant_perspective"] = row[19]
+        scores["pleasant_post_processing"] = row[20]
+        scores["pleasant_reflection"] = row[21]
+        scores["pleasant_symmetry"] = row[22]
+        scores["sharply_focused_subject"] = row[23]
+        scores["tastefully_blurred"] = row[24]
+        scores["well_chosen_subject"] = row[25]
+        scores["well_framed_subject"] = row[26]
+        scores["well_timed_shot"] = row[27]
+        photosdb._db_scoreinfo_uuid[uuid] = scores
+
+    conn.close()

osxphotos/photosdb/_photosdb_process_searchinfo.py

@@ -10,7 +10,7 @@ import uuid as uuidlib
 from pprint import pformat
 
 from .._constants import _PHOTOS_4_VERSION, SEARCH_CATEGORY_LABEL
-from ..utils import _db_is_locked, _debug, _open_sql_file
+from ..utils import _db_is_locked, _debug, _open_sql_file, normalize_unicode
 
 """
     This module should be imported in the class defintion of PhotosDB in photosdb.py
@@ -102,7 +102,7 @@ def _process_searchinfo(self):
     # 8: groups.lookup_identifier
 
     for row in c:
-        uuid = ints_to_uuid(row[1],row[2])
+        uuid = ints_to_uuid(row[1], row[2])
         # strings have null character appended, so strip it
         record = {}
         record["uuid"] = uuid
@@ -112,8 +112,8 @@ def _process_searchinfo(self):
         record["groupid"] = row[3]
         record["category"] = row[4]
         record["owning_groupid"] = row[5]
-        record["content_string"] = row[6].replace("\x00", "")
+        record["content_string"] = normalize_unicode(row[6].replace("\x00", ""))
-        record["normalized_string"] = row[7].replace("\x00", "")
+        record["normalized_string"] = normalize_unicode(row[7].replace("\x00", ""))
         record["lookup_identifier"] = row[8]
 
         try:
@@ -123,13 +123,9 @@ def _process_searchinfo(self):
 
         category = record["category"]
         try:
-            _db_searchinfo_categories[category].append(
+            _db_searchinfo_categories[category].append(record["normalized_string"])
-                record["normalized_string"]
-            )
         except KeyError:
-            _db_searchinfo_categories[category] = [
+            _db_searchinfo_categories[category] = [record["normalized_string"]]
-                record["normalized_string"]
-            ]
 
         if category == SEARCH_CATEGORY_LABEL:
             label = record["content_string"]
@@ -152,6 +148,8 @@ def _process_searchinfo(self):
             + pformat(self._db_searchinfo_labels_normalized)
         )
 
+    conn.close()
+
 
 @property
 def labels(self):
@@ -198,6 +196,7 @@ def labels_normalized_as_dict(self):
 
 # The following method is not imported into PhotosDB
 
+
 @lru_cache(maxsize=128)
 def ints_to_uuid(uuid_0, uuid_1):
     """ convert two signed ints into a UUID strings

osxphotos/photosdb/photosdb_utils.py

@@ -0,0 +1,84 @@
+""" utility functions used by PhotosDB """
+
+import logging
+import plistlib
+
+from .._constants import (
+    _PHOTOS_5_MODEL_VERSION,
+    _PHOTOS_6_MODEL_VERSION,
+    _TESTED_DB_VERSIONS,
+)
+from ..utils import _open_sql_file
+
+
+def get_db_version(db_file):
+    """ Gets the Photos DB version from LiGlobals table
+
+    Args:
+        db_file: path to photos.db database file containing LiGlobals table
+
+    Returns: version as str
+    """
+
+    version = None
+
+    (conn, c) = _open_sql_file(db_file)
+
+    # get database version
+    c.execute("SELECT value from LiGlobals where LiGlobals.keyPath is 'libraryVersion'")
+    version = c.fetchone()[0]
+    conn.close()
+
+    if version not in _TESTED_DB_VERSIONS:
+        print(
+            f"WARNING: Only tested on database versions [{', '.join(_TESTED_DB_VERSIONS)}]"
+            + f" You have database version={version} which has not been tested"
+        )
+
+    return version
+
+
+def get_model_version(db_file):
+    """ Returns the database model version from Z_METADATA
+    
+    Args:
+        db_file: path to Photos.sqlite database file containing Z_METADATA table
+        
+    Returns: model version as str
+    """
+
+    version = None
+
+    (conn, c) = _open_sql_file(db_file)
+
+    # get database version
+    c.execute("SELECT MAX(Z_VERSION), Z_PLIST FROM Z_METADATA")
+    results = c.fetchone()
+
+    conn.close()
+
+    plist = plistlib.loads(results[1])
+    return plist["PLModelVersion"]
+
+
+def get_db_model_version(db_file):
+    """ Returns Photos version based on model version found in db_file
+    
+    Args:
+        db_file: path to Photos.sqlite file
+    
+    Returns: int of major Photos version number (e.g. 5 or 6).
+    If unknown model version found, logs warning and returns most current Photos version.
+    """
+
+    model_ver = get_model_version(db_file)
+    if _PHOTOS_5_MODEL_VERSION[0] <= model_ver <= _PHOTOS_5_MODEL_VERSION[1]:
+        db_ver = 5
+    elif _PHOTOS_6_MODEL_VERSION[0] <= model_ver <= _PHOTOS_6_MODEL_VERSION[1]:
+        db_ver = 6
+    else:
+        logging.warning(f"Unknown model version: {model_ver}")
+        # cross our fingers and try latest version
+        db_ver = 6
+
+    return db_ver

osxphotos/phototemplate.py

@@ -9,13 +9,16 @@
 # 4. Couldn't figure out how to do #1 and #2 with str.format()
 #
 # This code isn't elegant but it seems to work well.  PRs gladly accepted.
+import datetime
 import locale
 import os
-import re
 import pathlib
+import re
+from functools import partial
 
 from ._constants import _UNKNOWN_PERSON
 from .datetime_formatter import DateTimeFormatter
+from .path_utils import sanitize_dirname, sanitize_filename, sanitize_pathpart
 
 # ensure locale set to user's locale
 locale.setlocale(locale.LC_ALL, "")
@@ -27,38 +30,56 @@ TEMPLATE_SUBSTITUTIONS = {
     "{title}": "Title of the photo",
     "{descr}": "Description of the photo",
     "{created.date}": "Photo's creation date in ISO format, e.g. '2020-03-22'",
-    "{created.year}": "4-digit year of file creation time",
+    "{created.year}": "4-digit year of photo creation time",
-    "{created.yy}": "2-digit year of file creation time",
+    "{created.yy}": "2-digit year of photo creation time",
-    "{created.mm}": "2-digit month of the file creation time (zero padded)",
+    "{created.mm}": "2-digit month of the photo creation time (zero padded)",
-    "{created.month}": "Month name in user's locale of the file creation time",
+    "{created.month}": "Month name in user's locale of the photo creation time",
-    "{created.mon}": "Month abbreviation in the user's locale of the file creation time",
+    "{created.mon}": "Month abbreviation in the user's locale of the photo creation time",
-    "{created.dd}": "2-digit day of the month (zero padded) of file creation time",
+    "{created.dd}": "2-digit day of the month (zero padded) of photo creation time",
-    "{created.dow}": "Day of week in user's locale of the file creation time",
+    "{created.dow}": "Day of week in user's locale of the photo creation time",
-    "{created.doy}": "3-digit day of year (e.g Julian day) of file creation time, starting from 1 (zero padded)",
+    "{created.doy}": "3-digit day of year (e.g Julian day) of photo creation time, starting from 1 (zero padded)",
-    "{created.hour}": "2-digit hour of the file creation time",
+    "{created.hour}": "2-digit hour of the photo creation time",
-    "{created.min}": "2-digit minute of the file creation time",
+    "{created.min}": "2-digit minute of the photo creation time",
-    "{created.sec}": "2-digit second of the file creation time",
+    "{created.sec}": "2-digit second of the photo creation time",
     "{created.strftime}": "Apply strftime template to file creation date/time. Should be used in form "
     + "{created.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. "
     + "{created.strftime,%Y-%U} would result in year-week number of year: '2020-23'. "
     + "If used with no template will return null value. "
     + "See https://strftime.org/ for help on strftime templates.",
     "{modified.date}": "Photo's modification date in ISO format, e.g. '2020-03-22'",
-    "{modified.year}": "4-digit year of file modification time",
+    "{modified.year}": "4-digit year of photo modification time",
-    "{modified.yy}": "2-digit year of file modification time",
+    "{modified.yy}": "2-digit year of photo modification time",
-    "{modified.mm}": "2-digit month of the file modification time (zero padded)",
+    "{modified.mm}": "2-digit month of the photo modification time (zero padded)",
-    "{modified.month}": "Month name in user's locale of the file modification time",
+    "{modified.month}": "Month name in user's locale of the photo modification time",
-    "{modified.mon}": "Month abbreviation in the user's locale of the file modification time",
+    "{modified.mon}": "Month abbreviation in the user's locale of the photo modification time",
-    "{modified.dd}": "2-digit day of the month (zero padded) of the file modification time",
+    "{modified.dd}": "2-digit day of the month (zero padded) of the photo modification time",
-    "{modified.doy}": "3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)",
+    "{modified.dow}": "Day of week in user's locale of the photo modification time",
-    "{modified.hour}": "2-digit hour of the file modification time",
+    "{modified.doy}": "3-digit day of year (e.g Julian day) of photo modification time, starting from 1 (zero padded)",
-    "{modified.min}": "2-digit minute of the file modification time",
+    "{modified.hour}": "2-digit hour of the photo modification time",
-    "{modified.sec}": "2-digit second of the file modification time",
+    "{modified.min}": "2-digit minute of the photo modification time",
+    "{modified.sec}": "2-digit second of the photo modification time",
     # "{modified.strftime}": "Apply strftime template to file modification date/time. Should be used in form "
     # + "{modified.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. "
     # + "{modified.strftime,%Y-%U} would result in year-week number of year: '2020-23'. "
     # + "If used with no template will return null value. "
     # + "See https://strftime.org/ for help on strftime templates.",
+    "{today.date}": "Current date in iso format, e.g. '2020-03-22'",
+    "{today.year}": "4-digit year of current date",
+    "{today.yy}": "2-digit year of current date",
+    "{today.mm}": "2-digit month of the current date (zero padded)",
+    "{today.month}": "Month name in user's locale of the current date",
+    "{today.mon}": "Month abbreviation in the user's locale of the current date",
+    "{today.dd}": "2-digit day of the month (zero padded) of current date",
+    "{today.dow}": "Day of week in user's locale of the current date",
+    "{today.doy}": "3-digit day of year (e.g Julian day) of current date, starting from 1 (zero padded)",
+    "{today.hour}": "2-digit hour of the current date",
+    "{today.min}": "2-digit minute of the current date",
+    "{today.sec}": "2-digit second of the current date",
+    "{today.strftime}": "Apply strftime template to current date/time. Should be used in form "
+    + "{today.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. "
+    + "{today.strftime,%Y-%U} would result in year-week number of year: '2020-23'. "
+    + "If used with no template will return null value. "
+    + "See https://strftime.org/ for help on strftime templates.",
     "{place.name}": "Place name from the photo's reverse geolocation data, as displayed in Photos",
     "{place.country_code}": "The ISO country code from the photo's reverse geolocation data",
     "{place.name.country}": "Country name from the photo's reverse geolocation data",
@@ -82,6 +103,7 @@ TEMPLATE_SUBSTITUTIONS_MULTI_VALUED = {
     "{person}": "Person(s) / face(s) in a photo",
     "{label}": "Image categorization label associated with a photo (Photos 5 only)",
     "{label_normalized}": "All lower case version of 'label' (Photos 5 only)",
+    "{comment}": "Comment(s) on shared Photos; format is 'Person name: comment text' (Photos 5 only)",
 }
 
 # Just the multi-valued substitution names without the braces
@@ -102,13 +124,34 @@ class PhotoTemplate:
         """
         self.photo = photo
 
-    def render(self, template, none_str="_", path_sep=None):
+        # holds value of current date/time for {today.x} fields
+        # gets initialized in get_template_value
+        self.today = None
+
+    def render(
+        self,
+        template,
+        none_str="_",
+        path_sep=None,
+        expand_inplace=False,
+        inplace_sep=None,
+        filename=False,
+        dirname=False,
+        replacement=":",
+    ):
         """ Render a filename or directory template 
 
         Args:
             template: str template 
             none_str: str to use default for None values, default is '_' 
             path_sep: optional character to use as path separator, default is os.path.sep
+            expand_inplace: expand multi-valued substitutions in-place as a single string 
+                instead of returning individual strings
+            inplace_sep: optional string to use as separator between multi-valued keywords
+            with expand_inplace; default is ','
+            filename: if True, template output will be sanitized to produce valid file name
+            dirname: if True, template output will be sanitized to produce valid directory name 
+            replacement: str, value to replace any illegal file path characters with; default = ":"
 
         Returns:
             ([rendered_strings], [unmatched]): tuple of list of rendered strings and list of unmatched template values
@@ -119,6 +162,9 @@ class PhotoTemplate:
         elif path_sep is not None and len(path_sep) != 1:
             raise ValueError(f"path_sep must be single character: {path_sep}")
 
+        if inplace_sep is None:
+            inplace_sep = ","
+
         # the rendering happens in two phases:
         # phase 1: handle all the single-value template substitutions
         #          results in a single string with all the template fields replaced
@@ -134,13 +180,21 @@ class PhotoTemplate:
         if type(template) is not str:
             raise TypeError(f"template must be type str, not {type(template)}")
 
-        def make_subst_function(self, none_str, get_func=self.get_template_value):
+        # used by make_subst_function to get the value for a template substitution
+        get_func = partial(
+            self.get_template_value,
+            filename=filename,
+            dirname=dirname,
+            replacement=replacement,
+        )
+
+        def make_subst_function(self, none_str, get_func=get_func):
             """ returns: substitution function for use in re.sub 
                 none_str: value to use if substitution lookup is None and no default provided
                 get_func: function that gets the substitution value for a given template field
                         default is get_template_value which handles the single-value fields """
 
-            # closure to capture photo, none_str in subst
+            # closure to capture photo, none_str, filename, dirname in subst
             def subst(matchobj):
                 groups = len(matchobj.groups())
                 if groups == 4:
@@ -150,13 +204,13 @@ class PhotoTemplate:
                         return matchobj.group(0)
 
                     if val is None:
-                        return (
+                        val = (
                             matchobj.group(3)
                             if matchobj.group(3) is not None
                             else none_str
                         )
-                    else:
+
-                        return val
+                    return val
                 else:
                     raise ValueError(
                         f"Unexpected number of groups: expected 4, got {groups}"
@@ -192,25 +246,37 @@ class PhotoTemplate:
         #    '2011/Album2/keyword1/person1',
         #    '2011/Album2/keyword2/person1',]
 
-        rendered_strings = set([rendered])
+        rendered_strings = [rendered]
         for field in MULTI_VALUE_SUBSTITUTIONS:
             # Build a regex that matches only the field being processed
             re_str = r"(?<!\\)\{(" + field + r")(,(([\w\-\%. ]{0,})))?\}"
             regex_multi = re.compile(re_str)
 
-            # holds each of the new rendered_strings, set() to avoid duplicates
+            # holds each of the new rendered_strings, dict to avoid repeats (dict.keys())
-            new_strings = set()
+            new_strings = {}
 
             for str_template in rendered_strings:
                 if regex_multi.search(str_template):
-                    values = self.get_template_value_multi(field, path_sep)
+                    values = self.get_template_value_multi(
-                    for val in values:
+                        field,
+                        path_sep,
+                        filename=filename,
+                        dirname=dirname,
+                        replacement=replacement,
+                    )
+                    if expand_inplace:
+                        # instead of returning multiple strings, join values into a single string
+                        val = (
+                            inplace_sep.join(sorted(values))
+                            if values and values[0]
+                            else None
+                        )
 
-                        def lookup_template_value_multi(lookup_value, default):
+                        def lookup_template_value_multi(lookup_value, _):
                             """ Closure passed to make_subst_function get_func 
-                                Capture val and field in the closure 
+                                    Capture val and field in the closure 
-                                Allows make_subst_function to be re-used w/o modification
+                                    Allows make_subst_function to be re-used w/o modification
-                                default is not used but required so signature matches get_template_value """
+                                    _ is not used but required so signature matches get_template_value """
                             if lookup_value == field:
                                 return val
                             else:
@@ -220,10 +286,33 @@ class PhotoTemplate:
                             self, none_str, get_func=lookup_template_value_multi
                         )
                         new_string = regex_multi.sub(subst, str_template)
-                        new_strings.add(new_string)
 
-                    # update rendered_strings for the next field to process
+                        # update rendered_strings for the next field to process
-                    rendered_strings = new_strings
+                        rendered_strings = {new_string}
+                    else:
+                        # create a new template string for each value
+                        for val in values:
+
+                            def lookup_template_value_multi(lookup_value, _):
+                                """ Closure passed to make_subst_function get_func 
+                                    Capture val and field in the closure 
+                                    Allows make_subst_function to be re-used w/o modification
+                                    _ is not used but required so signature matches get_template_value """
+                                if lookup_value == field:
+                                    return val
+                                else:
+                                    raise ValueError(
+                                        f"Unexpected value: {lookup_value}"
+                                    )
+
+                            subst = make_subst_function(
+                                self, none_str, get_func=lookup_template_value_multi
+                            )
+                            new_string = regex_multi.sub(subst, str_template)
+                            new_strings[new_string] = 1
+
+                        # update rendered_strings for the next field to process
+                        rendered_strings = list(new_strings.keys())
 
         # find any {fields} that weren't replaced
         unmatched = []
@@ -242,14 +331,24 @@ class PhotoTemplate:
             for rendered_str in rendered_strings
         ]
 
+        if filename:
+            rendered_strings = [
+                sanitize_filename(rendered_str) for rendered_str in rendered_strings
+            ]
+
         return rendered_strings, unmatched
 
-    def get_template_value(self, field, default):
+    def get_template_value(
+        self, field, default, filename=False, dirname=False, replacement=":"
+    ):
         """lookup value for template field (single-value template substitutions)
 
         Args:
             field: template field to find value for.
             default: the default value provided by the user
+            filename: if True, template output will be sanitized to produce valid file name
+            dirname: if True, template output will be sanitized to produce valid directory name 
+            replacement: str, value to replace any illegal file path characters with; default = ":"
         
         Returns:
             The matching template value (which may be None).
@@ -258,244 +357,246 @@ class PhotoTemplate:
             ValueError if no rule exists for field.
         """
 
-        # must be a valid keyword
+        # initialize today with current date/time if needed
+        if self.today is None:
+            self.today = datetime.datetime.now()
+
+        value = None
+
+        # wouldn't a switch/case statement be nice...
         if field == "name":
-            return pathlib.Path(self.photo.filename).stem
+            value = pathlib.Path(self.photo.filename).stem
-
+        elif field == "original_name":
-        if field == "original_name":
+            value = pathlib.Path(self.photo.original_filename).stem
-            return pathlib.Path(self.photo.original_filename).stem
+        elif field == "title":
-
+            value = self.photo.title
-        if field == "title":
+        elif field == "descr":
-            return self.photo.title
+            value = self.photo.description
-
+        elif field == "created.date":
-        if field == "descr":
+            value = DateTimeFormatter(self.photo.date).date
-            return self.photo.description
+        elif field == "created.year":
-
+            value = DateTimeFormatter(self.photo.date).year
-        if field == "created.date":
+        elif field == "created.yy":
-            return DateTimeFormatter(self.photo.date).date
+            value = DateTimeFormatter(self.photo.date).yy
-
+        elif field == "created.mm":
-        if field == "created.year":
+            value = DateTimeFormatter(self.photo.date).mm
-            return DateTimeFormatter(self.photo.date).year
+        elif field == "created.month":
-
+            value = DateTimeFormatter(self.photo.date).month
-        if field == "created.yy":
+        elif field == "created.mon":
-            return DateTimeFormatter(self.photo.date).yy
+            value = DateTimeFormatter(self.photo.date).mon
-
+        elif field == "created.dd":
-        if field == "created.mm":
+            value = DateTimeFormatter(self.photo.date).dd
-            return DateTimeFormatter(self.photo.date).mm
+        elif field == "created.dow":
-
+            value = DateTimeFormatter(self.photo.date).dow
-        if field == "created.month":
+        elif field == "created.doy":
-            return DateTimeFormatter(self.photo.date).month
+            value = DateTimeFormatter(self.photo.date).doy
-
+        elif field == "created.hour":
-        if field == "created.mon":
+            value = DateTimeFormatter(self.photo.date).hour
-            return DateTimeFormatter(self.photo.date).mon
+        elif field == "created.min":
-
+            value = DateTimeFormatter(self.photo.date).min
-        if field == "created.dd":
+        elif field == "created.sec":
-            return DateTimeFormatter(self.photo.date).dd
+            value = DateTimeFormatter(self.photo.date).sec
-
+        elif field == "created.strftime":
-        if field == "created.dow":
-            return DateTimeFormatter(self.photo.date).dow
-
-        if field == "created.doy":
-            return DateTimeFormatter(self.photo.date).doy
-
-        if field == "created.hour":
-            return DateTimeFormatter(self.photo.date).hour
-
-        if field == "created.min":
-            return DateTimeFormatter(self.photo.date).min
-
-        if field == "created.sec":
-            return DateTimeFormatter(self.photo.date).sec
-
-        if field == "created.strftime":
             if default:
                 try:
-                    return self.photo.date.strftime(default)
+                    value = self.photo.date.strftime(default)
                 except:
                     raise ValueError(f"Invalid strftime template: '{default}'")
             else:
-                return None
+                value = None
-
+        elif field == "modified.date":
-        if field == "modified.date":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).date
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.year":
-        if field == "modified.year":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).year
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.yy":
-        if field == "modified.yy":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).yy
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.mm":
-        if field == "modified.mm":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).mm
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.month":
-        if field == "modified.month":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).month
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.mon":
-        if field == "modified.mon":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).mon
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.dd":
-        if field == "modified.dd":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).dd
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.dow":
-        if field == "modified.doy":
+            value = (
-            return (
+                DateTimeFormatter(self.photo.date_modified).dow
+                if self.photo.date_modified
+                else None
+            )
+        elif field == "modified.doy":
+            value = (
                 DateTimeFormatter(self.photo.date_modified).doy
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.hour":
-        if field == "modified.hour":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).hour
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.min":
-        if field == "modified.min":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).min
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "modified.sec":
-        if field == "modified.sec":
+            value = (
-            return (
                 DateTimeFormatter(self.photo.date_modified).sec
                 if self.photo.date_modified
                 else None
             )
-
+        elif field == "today.date":
-        # TODO: disabling modified.strftime for now because now clean way to pass
+            value = DateTimeFormatter(self.today).date
-        # a default value if modified time is None
+        elif field == "today.year":
-        # if field == "modified.strftime":
+            value = DateTimeFormatter(self.today).year
-        #     if default and self.photo.date_modified:
+        elif field == "today.yy":
-        #         try:
+            value = DateTimeFormatter(self.today).yy
-        #             return self.photo.date_modified.strftime(default)
+        elif field == "today.mm":
-        #         except:
+            value = DateTimeFormatter(self.today).mm
-        #             raise ValueError(f"Invalid strftime template: '{default}'")
+        elif field == "today.month":
-        #     else:
+            value = DateTimeFormatter(self.today).month
-        #         return None
+        elif field == "today.mon":
-
+            value = DateTimeFormatter(self.today).mon
-        if field == "place.name":
+        elif field == "today.dd":
-            return self.photo.place.name if self.photo.place else None
+            value = DateTimeFormatter(self.today).dd
-
+        elif field == "today.dow":
-        if field == "place.country_code":
+            value = DateTimeFormatter(self.today).dow
-            return self.photo.place.country_code if self.photo.place else None
+        elif field == "today.doy":
-
+            value = DateTimeFormatter(self.today).doy
-        if field == "place.name.country":
+        elif field == "today.hour":
-            return (
+            value = DateTimeFormatter(self.today).hour
+        elif field == "today.min":
+            value = DateTimeFormatter(self.today).min
+        elif field == "today.sec":
+            value = DateTimeFormatter(self.today).sec
+        elif field == "today.strftime":
+            if default:
+                try:
+                    value = self.today.strftime(default)
+                except:
+                    raise ValueError(f"Invalid strftime template: '{default}'")
+            else:
+                value = None
+        elif field == "place.name":
+            value = self.photo.place.name if self.photo.place else None
+        elif field == "place.country_code":
+            value = self.photo.place.country_code if self.photo.place else None
+        elif field == "place.name.country":
+            value = (
                 self.photo.place.names.country[0]
                 if self.photo.place and self.photo.place.names.country
                 else None
             )
-
+        elif field == "place.name.state_province":
-        if field == "place.name.state_province":
+            value = (
-            return (
                 self.photo.place.names.state_province[0]
                 if self.photo.place and self.photo.place.names.state_province
                 else None
             )
-
+        elif field == "place.name.city":
-        if field == "place.name.city":
+            value = (
-            return (
                 self.photo.place.names.city[0]
                 if self.photo.place and self.photo.place.names.city
                 else None
             )
-
+        elif field == "place.name.area_of_interest":
-        if field == "place.name.area_of_interest":
+            value = (
-            return (
                 self.photo.place.names.area_of_interest[0]
                 if self.photo.place and self.photo.place.names.area_of_interest
                 else None
             )
-
+        elif field == "place.address":
-        if field == "place.address":
+            value = (
-            return (
                 self.photo.place.address_str
                 if self.photo.place and self.photo.place.address_str
                 else None
             )
-
+        elif field == "place.address.street":
-        if field == "place.address.street":
+            value = (
-            return (
                 self.photo.place.address.street
                 if self.photo.place and self.photo.place.address.street
                 else None
             )
-
+        elif field == "place.address.city":
-        if field == "place.address.city":
+            value = (
-            return (
                 self.photo.place.address.city
                 if self.photo.place and self.photo.place.address.city
                 else None
             )
-
+        elif field == "place.address.state_province":
-        if field == "place.address.state_province":
+            value = (
-            return (
                 self.photo.place.address.state_province
                 if self.photo.place and self.photo.place.address.state_province
                 else None
             )
-
+        elif field == "place.address.postal_code":
-        if field == "place.address.postal_code":
+            value = (
-            return (
                 self.photo.place.address.postal_code
                 if self.photo.place and self.photo.place.address.postal_code
                 else None
             )
-
+        elif field == "place.address.country":
-        if field == "place.address.country":
+            value = (
-            return (
                 self.photo.place.address.country
                 if self.photo.place and self.photo.place.address.country
                 else None
             )
-
+        elif field == "place.address.country_code":
-        if field == "place.address.country_code":
+            value = (
-            return (
                 self.photo.place.address.iso_country_code
                 if self.photo.place and self.photo.place.address.iso_country_code
                 else None
             )
+        else:
+            # if here, didn't get a match
+            raise ValueError(f"Unhandled template value: {field}")
 
-        # if here, didn't get a match
+        if filename:
-        raise ValueError(f"Unhandled template value: {field}")
+            value = sanitize_pathpart(value, replacement=replacement)
+        elif dirname:
+            value = sanitize_dirname(value, replacement=replacement)
+        return value
 
-    def get_template_value_multi(self, field, path_sep):
+    def get_template_value_multi(
+        self, field, path_sep, filename=False, dirname=False, replacement=":"
+    ):
         """lookup value for template field (multi-value template substitutions)
 
         Args:
             field: template field to find value for.
             path_sep: path separator to use for folder_album field
+            dirname: if True, values will be sanitized to be valid directory names; default = False
         
         Returns:
             List of the matching template values or [None].
@@ -523,15 +624,46 @@ class PhotoTemplate:
             for album in self.photo.album_info:
                 if album.folder_names:
                     # album in folder
-                    folder = path_sep.join(album.folder_names)
+                    if dirname:
-                    folder += path_sep + album.title
+                        # being used as a filepath so sanitize each part
+                        folder = path_sep.join(
+                            sanitize_dirname(f, replacement=replacement)
+                            for f in album.folder_names
+                        )
+                        folder += path_sep + sanitize_dirname(
+                            album.title, replacement=replacement
+                        )
+                    else:
+                        folder = path_sep.join(album.folder_names)
+                        folder += path_sep + album.title
                     values.append(folder)
                 else:
                     # album not in folder
-                    values.append(album.title)
+                    if dirname:
+                        values.append(
+                            sanitize_dirname(album.title, replacement=replacement)
+                        )
+                    else:
+                        values.append(album.title)
+        elif field == "comment":
+            values = [
+                f"{comment.user}: {comment.text}" for comment in self.photo.comments
+            ]
         else:
-            raise ValueError(f"Unhandleded template value: {field}")
+            raise ValueError(f"Unhandled template value: {field}")
+
+        # sanitize directory names if needed, folder_album handled differently above
+        if filename:
+            values = [
+                sanitize_pathpart(value, replacement=replacement) for value in values
+            ]
+        elif dirname and field != "folder_album":
+            # skip folder_album because it would have been handled above
+            values = [
+                sanitize_dirname(value, replacement=replacement) for value in values
+            ]
 
         # If no values, insert None so code below will substite none_str for None
         values = values or [None]
         return values
+

osxphotos/placeinfo.py

@@ -11,6 +11,9 @@ from collections import namedtuple  # pylint: disable=syntax-error
 import yaml
 from bpylist import archiver
 
+from ._constants import UNICODE_FORMAT
+from .utils import normalize_unicode
+
 # postal address information, returned by PlaceInfo.address
 PostalAddress = namedtuple(
     "PostalAddress",
@@ -76,12 +79,12 @@ class PLRevGeoLocationInfo:
         geoServiceProvider,
         postalAddress,
     ):
-        self.addressString = addressString
+        self.addressString = normalize_unicode(addressString)
         self.countryCode = countryCode
         self.mapItem = mapItem
         self.isHome = isHome
-        self.compoundNames = compoundNames
+        self.compoundNames = normalize_unicode(compoundNames)
-        self.compoundSecondaryNames = compoundSecondaryNames
+        self.compoundSecondaryNames = normalize_unicode(compoundSecondaryNames)
         self.version = version
         self.geoServiceProvider = geoServiceProvider
         self.postalAddress = postalAddress
@@ -183,7 +186,7 @@ class PLRevGeoMapItemAdditionalPlaceInfo:
 
     def __init__(self, area, name, placeType, dominantOrderType):
         self.area = area
-        self.name = name
+        self.name = normalize_unicode(name)
         self.placeType = placeType
         self.dominantOrderType = dominantOrderType
 
@@ -232,13 +235,13 @@ class CNPostalAddress:
         _subLocality,
     ):
         self._ISOCountryCode = _ISOCountryCode
-        self._city = _city
+        self._city = normalize_unicode(_city)
-        self._country = _country
+        self._country = normalize_unicode(_country)
-        self._postalCode = _postalCode
+        self._postalCode = normalize_unicode(_postalCode)
-        self._state = _state
+        self._state = normalize_unicode(_state)
-        self._street = _street
+        self._street = normalize_unicode(_street)
-        self._subAdministrativeArea = _subAdministrativeArea
+        self._subAdministrativeArea = normalize_unicode(_subAdministrativeArea)
-        self._subLocality = _subLocality
+        self._subLocality = normalize_unicode(_subLocality)
 
     def __eq__(self, other):
         return all(
@@ -414,9 +417,9 @@ class PlaceInfo4(PlaceInfo):
             #   2: type
             #   3: area
             try:
-                places_dict[p[2]].append((p[1], p[3]))
+                places_dict[p[2]].append((normalize_unicode(p[1]), p[3]))
             except KeyError:
-                places_dict[p[2]] = [(p[1], p[3])]
+                places_dict[p[2]] = [(normalize_unicode(p[1]), p[3])]
 
         # build list to populate PlaceNames tuple
         # initialize with empty lists for each field in PlaceNames
@@ -488,7 +491,7 @@ class PlaceInfo4(PlaceInfo):
         }
         return "PlaceInfo(" + ", ".join([f"{k}='{v}'" for k, v in info.items()]) + ")"
 
-    def as_dict(self):
+    def asdict(self):
         return {
             "name": self.name,
             "names": self.names._asdict(),
@@ -503,7 +506,6 @@ class PlaceInfo5(PlaceInfo):
         """ revgeoloc_bplist: a binary plist blob containing 
                 a serialized PLRevGeoLocationInfo object """
         self._bplist = revgeoloc_bplist
-        # todo: check for None?
         self._plrevgeoloc = archiver.unarchive(revgeoloc_bplist)
         self._process_place_info()
 
@@ -535,16 +537,23 @@ class PlaceInfo5(PlaceInfo):
     @property
     def address(self):
         addr = self._plrevgeoloc.postalAddress
-        return PostalAddress(
+        if addr is not None:
-            street=addr._street,
+            postal_address = PostalAddress(
-            sub_locality=addr._subLocality,
+                street=addr._street,
-            city=addr._city,
+                sub_locality=addr._subLocality,
-            sub_administrative_area=addr._subAdministrativeArea,
+                city=addr._city,
-            state_province=addr._state,
+                sub_administrative_area=addr._subAdministrativeArea,
-            postal_code=addr._postalCode,
+                state_province=addr._state,
-            country=addr._country,
+                postal_code=addr._postalCode,
-            iso_country_code=addr._ISOCountryCode,
+                country=addr._country,
-        )
+                iso_country_code=addr._ISOCountryCode,
+            )
+        else:
+            postal_address = PostalAddress(
+                None, None, None, None, None, None, None, None
+            )
+
+        return postal_address
 
     def _process_place_info(self):
         """ Process sortedPlaceInfos to set self._name and self._names """
@@ -625,12 +634,12 @@ class PlaceInfo5(PlaceInfo):
         }
         return "PlaceInfo(" + ", ".join([f"{k}='{v}'" for k, v in info.items()]) + ")"
 
-    def as_dict(self):
+    def asdict(self):
         return {
             "name": self.name,
             "names": self.names._asdict(),
             "country_code": self.country_code,
             "ishome": self.ishome,
             "address_str": self.address_str,
-            "address": self.address._asdict(),
+            "address": self.address._asdict() if self.address is not None else None,
         }

osxphotos/templates/xmp_sidecar.mako

@@ -1,5 +1,13 @@
 <!-- Created with osxphotos https://github.com/RhetTbull/osxphotos -->
 
+<%def name="photoshop_sidecar_for_extension(extension)">
+    % if extension is None:
+        <photoshop:SidecarForExtension></photoshop:SidecarForExtension>
+    % else:
+        <photoshop:SidecarForExtension>${extension}</photoshop:SidecarForExtension>
+    % endif
+</%def>
+
 <%def name="dc_description(desc)">
     % if desc is None:
         <dc:description></dc:description>
@@ -71,29 +79,43 @@
     % endif
 </%def>
 
+<%def name="gps_info(latitude, longitude)">
+    % if latitude is not None and longitude is not None:
+        <exif:GPSLongitudeRef>${"E" if longitude >= 0 else "W"}</exif:GPSLongitudeRef>
+        <exif:GPSLongitude>${abs(longitude)}</exif:GPSLongitude>
+        <exif:GPSLatitude>${abs(latitude)}</exif:GPSLatitude>
+        <exif:GPSLatitudeRef>${"N" if latitude >= 0 else "S"}</exif:GPSLatitudeRef>
+    % endif
+</%def>
+
 <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
     <!-- mirrors Photos 5 "Export IPTC as XMP" option -->
     <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
         <rdf:Description rdf:about="" 
             xmlns:dc="http://purl.org/dc/elements/1.1/" 
             xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
-            ${dc_description(photo.description)}
+            ${photoshop_sidecar_for_extension(extension)}
+            ${dc_description(description)}
             ${dc_title(photo.title)}
             ${dc_subject(subjects)}
             ${dc_datecreated(photo.date)}
         </rdf:Description>
-        <rdf:Description rdf:about='' 
+        <rdf:Description rdf:about=""  
             xmlns:Iptc4xmpExt='http://iptc.org/std/Iptc4xmpExt/2008-02-29/'>
             ${iptc_personinimage(persons)}
         </rdf:Description>
-        <rdf:Description rdf:about='' 
+        <rdf:Description rdf:about="" 
             xmlns:digiKam='http://www.digikam.org/ns/1.0/'>
             ${dk_tagslist(keywords)}
         </rdf:Description>
-        <rdf:Description rdf:about='' 
+        <rdf:Description rdf:about="" 
             xmlns:xmp='http://ns.adobe.com/xap/1.0/'>
             ${adobe_createdate(photo.date)}
             ${adobe_modifydate(photo.date)}
         </rdf:Description>
+        <rdf:Description rdf:about=""
+            xmlns:exif='http://ns.adobe.com/exif/1.0/'>
+            ${gps_info(*photo.location)}
+        </rdf:Description>
    </rdf:RDF>
 </x:xmpmeta>

osxphotos/utils.py

@@ -10,6 +10,7 @@ import sqlite3
 import subprocess
 import sys
 import tempfile
+import unicodedata
 import urllib.parse
 from plistlib import load as plistload
 
@@ -18,6 +19,7 @@ import CoreServices
 import objc
 from Foundation import *
 
+from ._constants import UNICODE_FORMAT
 from .fileutil import FileUtil
 
 _DEBUG = False
@@ -55,6 +57,9 @@ def _debug():
     """ returns True if debugging turned on (via _set_debug), otherwise, false """
     return _DEBUG
 
+def noop(*args, **kwargs):
+    """ do nothing (no operation) """
+    pass
 
 def _get_os_version():
     # returns tuple containing OS version
@@ -149,7 +154,7 @@ def dd_to_dms_str(lat, lon):
 
 def get_system_library_path():
     """ return the path to the system Photos library as string """
-    """ only works on MacOS 10.15+ """
+    """ only works on MacOS 10.15 """
     """ on earlier versions, returns None """
     _, major, _ = _get_os_version()
     if int(major) < 15:
@@ -166,16 +171,10 @@ def get_system_library_path():
         with open(plist_file, "rb") as fp:
             pl = plistload(fp)
     else:
-        logging.warning(f"could not find plist file: {str(plist_file)}")
+        logging.debug(f"could not find plist file: {str(plist_file)}")
         return None
 
-    photospath = pl["SystemLibraryPath"]
+    return pl.get("SystemLibraryPath")
-
-    if photospath is not None:
-        return photospath
-    else:
-        logging.warning("Could not get path to Photos database")
-        return None
 
 
 def get_last_library_path():
@@ -194,7 +193,7 @@ def get_last_library_path():
 
     # get the IPXDefaultLibraryURLBookmark from com.apple.Photos.plist
     # this is a serialized CFData object
-    photosurlref = pl["IPXDefaultLibraryURLBookmark"]
+    photosurlref = pl.get("IPXDefaultLibraryURLBookmark")
 
     if photosurlref is not None:
         # use CFURLCreateByResolvingBookmarkData to de-serialize bookmark data into a CFURLRef
@@ -268,7 +267,10 @@ def get_preferred_uti_extension(uti):
 
 def findfiles(pattern, path_):
     """Returns list of filenames from path_ matched by pattern
-       shell pattern. Matching is case-insensitive."""
+       shell pattern. Matching is case-insensitive.
+       If 'path_' is invalid/doesn't exist, returns []."""
+    if not os.path.isdir(path_):
+        return []
     # See: https://gist.github.com/techtonik/5694830
 
     rule = re.compile(fnmatch.translate(pattern), re.IGNORECASE)
@@ -355,3 +357,13 @@ def _db_is_locked(dbname):
 #     attr = xattr.xattr(filepath)
 #     uuid_bytes = bytes(uuid, 'utf-8')
 #     attr.set(OSXPHOTOS_XATTR_UUID, uuid_bytes)
+
+
+def normalize_unicode(value):
+    """ normalize unicode data """
+    if value is not None:
+        if not isinstance(value, str):
+            raise ValueError("value must be str")
+        return unicodedata.normalize(UNICODE_FORMAT, value)
+    else:
+        return None

requirements.txt

@@ -1,19 +1,38 @@
+aiohttp==4.0.0a1
 altgraph==0.17
 ansimarkup==1.4.0
 appdirs==1.4.3
+appnope==0.1.0
 astroid==2.2.5
+async-timeout==3.0.1
 atomicwrites==1.3.0
 attrs==19.1.0
+backcall==0.1.0
 better-exceptions-fork==0.2.1.post6
-# bpylist2==2.0.3;python_version<"3.8"
+black==19.10b0
-https://github.com/RhetTbull/bpylist/releases/download/v2.0.3/bpylist2-2.0.3.tar.gz#egg=bpylist2;python_version<"3.8"
+bleach==3.1.4
-bpylist2==3.0.0;python_version>="3.8"
+bpylist2==3.0.2
-certifi==2019.3.9
+certifi==2020.4.5.1
+cffi==1.14.0
+chardet==3.0.4
 Click==7.0
 colorama==0.4.1
 coverage==4.5.4
-importlib-metadata>=0.18
+decorator==4.4.2
+distlib==0.3.1
+docutils==0.16
+entrypoints==0.3
+filelock==3.0.12
+idna==2.9
+importlib-metadata==1.6.0
+ipykernel==5.1.4
+ipython==7.13.0
+ipython-genutils==0.2.0
 isort==4.3.20
+jedi==0.16.0
+jupyter-client==6.1.2
+jupyter-core==4.6.3
+keyring==21.2.0
 lazy-object-proxy==1.4.1
 loguru==0.2.5
 macholib==1.14
@@ -22,135 +41,167 @@ MarkupSafe==1.1.1
 mccabe==0.6.1
 modulegraph==0.18
 more-itertools==7.2.0
+multidict==4.7.6
 packaging==19.0
+parso==0.6.2
 pathspec==0.7.0
 pathvalidate==2.2.1
+pexpect==4.8.0
+pickleshare==0.7.5
+Pillow==7.2.0
+pkginfo==1.5.0.1
 pluggy==0.12.0
+prompt-toolkit==3.0.4
+psutil==5.7.0
+ptyprocess==0.6.0
 py==1.8.0
 py2app==0.21
-Pygments==2.4.2
+pycparser==2.20
+pyfiglet==0.8.post1
+Pygments==2.6.1
+PyInstaller==3.6
+pyinstaller-setuptools==2019.3
 pylint==2.3.1
-pyobjc==6.0.1
+pyobjc==6.2.2
-pyobjc-core==6.0.1
+pyobjc-core==6.2.2
-pyobjc-framework-Accounts==6.0.1
+pyobjc-framework-Accounts==6.2.2
-pyobjc-framework-AddressBook==6.0.1
+pyobjc-framework-AddressBook==6.2.2
-pyobjc-framework-AdSupport==6.0.1
+pyobjc-framework-AdSupport==6.2.2
-pyobjc-framework-AppleScriptKit==6.0.1
+pyobjc-framework-AppleScriptKit==6.2.2
-pyobjc-framework-AppleScriptObjC==6.0.1
+pyobjc-framework-AppleScriptObjC==6.2.2
-pyobjc-framework-ApplicationServices==6.0.1
+pyobjc-framework-ApplicationServices==6.2.2
-pyobjc-framework-AuthenticationServices==6.0.1
+pyobjc-framework-AuthenticationServices==6.2.2
-pyobjc-framework-Automator==6.0.1
+pyobjc-framework-AutomaticAssessmentConfiguration==6.2.2
-pyobjc-framework-AVFoundation==6.0.1
+pyobjc-framework-Automator==6.2.2
-pyobjc-framework-AVKit==6.0.1
+pyobjc-framework-AVFoundation==6.2.2
-pyobjc-framework-BusinessChat==6.0.1
+pyobjc-framework-AVKit==6.2.2
-pyobjc-framework-CalendarStore==6.0.1
+pyobjc-framework-BusinessChat==6.2.2
-pyobjc-framework-CFNetwork==6.0.1
+pyobjc-framework-CalendarStore==6.2.2
-pyobjc-framework-CloudKit==6.0.1
+pyobjc-framework-CFNetwork==6.2.2
-pyobjc-framework-Cocoa==6.0.1
+pyobjc-framework-CloudKit==6.2.2
-pyobjc-framework-Collaboration==6.0.1
+pyobjc-framework-Cocoa==6.2.2
-pyobjc-framework-ColorSync==6.0.1
+pyobjc-framework-Collaboration==6.2.2
-pyobjc-framework-Contacts==6.0.1
+pyobjc-framework-ColorSync==6.2.2
-pyobjc-framework-ContactsUI==6.0.1
+pyobjc-framework-Contacts==6.2.2
-pyobjc-framework-CoreAudio==6.0.1
+pyobjc-framework-ContactsUI==6.2.2
-pyobjc-framework-CoreAudioKit==6.0.1
+pyobjc-framework-CoreAudio==6.2.2
-pyobjc-framework-CoreBluetooth==6.0.1
+pyobjc-framework-CoreAudioKit==6.2.2
-pyobjc-framework-CoreData==6.0.1
+pyobjc-framework-CoreBluetooth==6.2.2
-pyobjc-framework-CoreHaptics==6.0.1
+pyobjc-framework-CoreData==6.2.2
-pyobjc-framework-CoreLocation==6.0.1
+pyobjc-framework-CoreHaptics==6.2.2
-pyobjc-framework-CoreMedia==6.0.1
+pyobjc-framework-CoreLocation==6.2.2
-pyobjc-framework-CoreMediaIO==6.0.1
+pyobjc-framework-CoreMedia==6.2.2
-pyobjc-framework-CoreML==6.0.1
+pyobjc-framework-CoreMediaIO==6.2.2
-pyobjc-framework-CoreMotion==6.0.1
+pyobjc-framework-CoreML==6.2.2
-pyobjc-framework-CoreServices==6.0.1
+pyobjc-framework-CoreMotion==6.2.2
-pyobjc-framework-CoreSpotlight==6.0.1
+pyobjc-framework-CoreServices==6.2.2
-pyobjc-framework-CoreText==6.0.1
+pyobjc-framework-CoreSpotlight==6.2.2
-pyobjc-framework-CoreWLAN==6.0.1
+pyobjc-framework-CoreText==6.2.2
-pyobjc-framework-CryptoTokenKit==6.0.1
+pyobjc-framework-CoreWLAN==6.2.2
-pyobjc-framework-DeviceCheck==6.0.1
+pyobjc-framework-CryptoTokenKit==6.2.2
-pyobjc-framework-DictionaryServices==6.0.1
+pyobjc-framework-DeviceCheck==6.2.2
-pyobjc-framework-DiscRecording==6.0.1
+pyobjc-framework-DictionaryServices==6.2.2
-pyobjc-framework-DiscRecordingUI==6.0.1
+pyobjc-framework-DiscRecording==6.2.2
-pyobjc-framework-DiskArbitration==6.0.1
+pyobjc-framework-DiscRecordingUI==6.2.2
-pyobjc-framework-DVDPlayback==6.0.1
+pyobjc-framework-DiskArbitration==6.2.2
-pyobjc-framework-EventKit==6.0.1
+pyobjc-framework-DVDPlayback==6.2.2
-pyobjc-framework-ExceptionHandling==6.0.1
+pyobjc-framework-EventKit==6.2.2
-pyobjc-framework-ExecutionPolicy==6.0.1
+pyobjc-framework-ExceptionHandling==6.2.2
-pyobjc-framework-ExternalAccessory==6.0.1
+pyobjc-framework-ExecutionPolicy==6.2.2
-pyobjc-framework-FileProvider==6.0.1
+pyobjc-framework-ExternalAccessory==6.2.2
-pyobjc-framework-FileProviderUI==6.0.1
+pyobjc-framework-FileProvider==6.2.2
-pyobjc-framework-FinderSync==6.0.1
+pyobjc-framework-FileProviderUI==6.2.2
-pyobjc-framework-FSEvents==6.0.1
+pyobjc-framework-FinderSync==6.2.2
-pyobjc-framework-GameCenter==6.0.1
+pyobjc-framework-FSEvents==6.2.2
-pyobjc-framework-GameController==6.0.1
+pyobjc-framework-GameCenter==6.2.2
-pyobjc-framework-GameKit==6.0.1
+pyobjc-framework-GameController==6.2.2
-pyobjc-framework-GameplayKit==6.0.1
+pyobjc-framework-GameKit==6.2.2
-pyobjc-framework-ImageCaptureCore==6.0.1
+pyobjc-framework-GameplayKit==6.2.2
-pyobjc-framework-IMServicePlugIn==6.0.1
+pyobjc-framework-ImageCaptureCore==6.2.2
-pyobjc-framework-InputMethodKit==6.0.1
+pyobjc-framework-IMServicePlugIn==6.2.2
-pyobjc-framework-InstallerPlugins==6.0.1
+pyobjc-framework-InputMethodKit==6.2.2
-pyobjc-framework-InstantMessage==6.0.1
+pyobjc-framework-InstallerPlugins==6.2.2
-pyobjc-framework-Intents==6.0.1
+pyobjc-framework-InstantMessage==6.2.2
-pyobjc-framework-IOSurface==6.0.1
+pyobjc-framework-Intents==6.2.2
-pyobjc-framework-iTunesLibrary==6.0.1
+pyobjc-framework-IOSurface==6.2.2
-pyobjc-framework-LatentSemanticMapping==6.0.1
+pyobjc-framework-iTunesLibrary==6.2.2
-pyobjc-framework-LaunchServices==6.0.1
+pyobjc-framework-LatentSemanticMapping==6.2.2
-pyobjc-framework-libdispatch==6.0.1
+pyobjc-framework-LaunchServices==6.2.2
-pyobjc-framework-LinkPresentation==6.0.1
+pyobjc-framework-libdispatch==6.2.2
-pyobjc-framework-LocalAuthentication==6.0.1
+pyobjc-framework-LinkPresentation==6.2.2
-pyobjc-framework-MapKit==6.0.1
+pyobjc-framework-LocalAuthentication==6.2.2
-pyobjc-framework-MediaAccessibility==6.0.1
+pyobjc-framework-MapKit==6.2.2
-pyobjc-framework-MediaLibrary==6.0.1
+pyobjc-framework-MediaAccessibility==6.2.2
-pyobjc-framework-MediaPlayer==6.0.1
+pyobjc-framework-MediaLibrary==6.2.2
-pyobjc-framework-MediaToolbox==6.0.1
+pyobjc-framework-MediaPlayer==6.2.2
-pyobjc-framework-MetalKit==6.0.1
+pyobjc-framework-MediaToolbox==6.2.2
-pyobjc-framework-ModelIO==6.0.1
+pyobjc-framework-Metal==6.2.2
-pyobjc-framework-MultipeerConnectivity==6.0.1
+pyobjc-framework-MetalKit==6.2.2
-pyobjc-framework-NaturalLanguage==6.0.1
+pyobjc-framework-ModelIO==6.2.2
-pyobjc-framework-NetFS==6.0.1
+pyobjc-framework-MultipeerConnectivity==6.2.2
-pyobjc-framework-Network==6.0.1
+pyobjc-framework-NaturalLanguage==6.2.2
-pyobjc-framework-NetworkExtension==6.0.1
+pyobjc-framework-NetFS==6.2.2
-pyobjc-framework-NotificationCenter==6.0.1
+pyobjc-framework-Network==6.2.2
-pyobjc-framework-OpenDirectory==6.0.1
+pyobjc-framework-NetworkExtension==6.2.2
-pyobjc-framework-OSAKit==6.0.1
+pyobjc-framework-NotificationCenter==6.2.2
-pyobjc-framework-OSLog==6.0.1
+pyobjc-framework-OpenDirectory==6.2.2
-pyobjc-framework-PencilKit==6.0.1
+pyobjc-framework-OSAKit==6.2.2
-pyobjc-framework-Photos==6.0.1
+pyobjc-framework-OSLog==6.2.2
-pyobjc-framework-PhotosUI==6.0.1
+pyobjc-framework-PencilKit==6.2.2
-pyobjc-framework-PreferencePanes==6.0.1
+pyobjc-framework-Photos==6.2.2
-pyobjc-framework-PubSub==6.0.1
+pyobjc-framework-PhotosUI==6.2.2
-pyobjc-framework-PushKit==6.0.1
+pyobjc-framework-PreferencePanes==6.2.2
+pyobjc-framework-PubSub==6.2
+pyobjc-framework-PushKit==6.2.2
 pyobjc-framework-QTKit==6.0.1
-pyobjc-framework-Quartz==6.0.1
+pyobjc-framework-Quartz==6.2.2
-pyobjc-framework-QuickLookThumbnailing==6.0.1
+pyobjc-framework-QuickLookThumbnailing==6.2.2
-pyobjc-framework-SafariServices==6.0.1
+pyobjc-framework-SafariServices==6.2.2
-pyobjc-framework-SceneKit==6.0.1
+pyobjc-framework-SceneKit==6.2.2
-pyobjc-framework-ScreenSaver==6.0.1
+pyobjc-framework-ScreenSaver==6.2.2
-pyobjc-framework-ScriptingBridge==6.0.1
+pyobjc-framework-ScriptingBridge==6.2.2
-pyobjc-framework-SearchKit==6.0.1
+pyobjc-framework-SearchKit==6.2.2
-pyobjc-framework-Security==6.0.1
+pyobjc-framework-Security==6.2.2
-pyobjc-framework-SecurityFoundation==6.0.1
+pyobjc-framework-SecurityFoundation==6.2.2
-pyobjc-framework-SecurityInterface==6.0.1
+pyobjc-framework-SecurityInterface==6.2.2
-pyobjc-framework-ServiceManagement==6.0.1
+pyobjc-framework-ServiceManagement==6.2.2
-pyobjc-framework-Social==6.0.1
+pyobjc-framework-Social==6.2.2
-pyobjc-framework-SoundAnalysis==6.0.1
+pyobjc-framework-SoundAnalysis==6.2.2
-pyobjc-framework-Speech==6.0.1
+pyobjc-framework-Speech==6.2.2
-pyobjc-framework-SpriteKit==6.0.1
+pyobjc-framework-SpriteKit==6.2.2
-pyobjc-framework-StoreKit==6.0.1
+pyobjc-framework-StoreKit==6.2.2
-pyobjc-framework-SyncServices==6.0.1
+pyobjc-framework-SyncServices==6.2.2
-pyobjc-framework-SystemConfiguration==6.0.1
+pyobjc-framework-SystemConfiguration==6.2.2
-pyobjc-framework-SystemExtensions==6.0.1
+pyobjc-framework-SystemExtensions==6.2.2
-pyobjc-framework-UserNotifications==6.0.1
+pyobjc-framework-UserNotifications==6.2.2
-pyobjc-framework-VideoSubscriberAccount==6.0.1
+pyobjc-framework-VideoSubscriberAccount==6.2.2
-pyobjc-framework-VideoToolbox==6.0.1
+pyobjc-framework-VideoToolbox==6.2.2
-pyobjc-framework-Vision==6.0.1
+pyobjc-framework-Vision==6.2.2
-pyobjc-framework-WebKit==6.0.1
+pyobjc-framework-WebKit==6.2.2
 pyparsing==2.4.1.1
+python-dateutil==2.8.1
 PyYAML==5.1.2
+pyzmq==18.1.1
+readme-renderer==25.0
 regex==2020.2.20
-six==1.12.0
+requests==2.23.0
+requests-toolbelt==0.9.1
+six==1.14.0
 termcolor==1.1.0
 toml==0.10.0
+tornado==6.0.4
+tox==3.19.0
+tox-conda==0.2.1
+tqdm==4.45.0
+traitlets==4.3.3
+twine==3.1.1
 typed-ast==1.4.1
-wcwidth==0.1.7
+typing-extensions==3.7.4.2
+urllib3==1.25.9
+virtualenv==20.0.30
+wcwidth==0.1.9
+webencodings==0.5.1
 wrapt==1.11.1
+wurlitzer==2.0.1
+yarl==1.4.2
 zipp==0.5.2

setup.py

@@ -48,17 +48,6 @@ with open(
 with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
     about["long_description"] = f.read()
 
-# ugly hack to install custom version of bpylist2 needed for Python < 3.8
-# the stock version of bylist2==2.0.3 causes an error related to
-# "pkg_resources.ContextualVersionConflict: (pycodestyle 2.3.1..."
-# PEP 508 no help here as URL-based lookups not allowed in PyPI packages
-# if you know a better way, PRs welcome!
-# once I go to 3.8+ required, this won't be necessary as bpylist2 3.0+ solves this issue
-if py_ver < 3.8:
-    os.system(
-        "python3 -m pip install git+git://github.com/RhetTbull/bpylist2.git#egg=bpylist2"
-    )
-
 setup(
     name="osxphotos",
     version=about["__version__"],
@@ -78,18 +67,18 @@ setup(
         "Intended Audience :: Developers",
         "License :: OSI Approved :: MIT License",
         "Operating System :: MacOS :: MacOS X",
-        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.7",
         "Topic :: Software Development :: Libraries :: Python Modules",
     ],
     install_requires=[
-        "pyobjc>=6.0.1",
+        "pyobjc>=6.2.2",
         "Click>=7",
         "PyYAML>=5.1.2",
         "Mako>=1.1.1",
-        "bpylist2==2.0.3;python_version<'3.8'",
+        "bpylist2==3.0.2",
-        "bpylist2==3.0.0;python_version>='3.8'",
         "pathvalidate==2.2.1",
         "dataclasses==0.7;python_version<'3.7'",
+        "wurlitzer>=2.0.1",
     ],
     entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]},
     include_package_data=True,

tests/README.md

@@ -17,14 +17,29 @@ Some of the export tests rely on photos in my local library and will look for `O
 
 One test for locale does not run on GitHub's automated workflow and will look for `OSXPHOTOS_TEST_LOCALE=1` to determine if it should be run.  If you want to run this test, set the environment variable.  
 
-## Attribution ##
+## Test Photo Libraries
-These tests utilize a test Photos library. The test library is populated with photos from [flickr](https://www.flickr.com).  All images used are licensed under Creative Commons 2.0 Attribution [license](https://creativecommons.org/licenses/by/2.0/).  
+**Important**: The test code uses several test photo libraries created on various version of MacOS.  If you need to inspect one of these or modify one for a test, make a copy of the library (for example, copy it to your ~/Pictures folder) then open the copy in Photos.  Once done, copy the revised library back to the tests/ folder.  If you do not do this, the Photos background process photoanalysisd will forever try to process the library resulting in updates to the database which will cause git to see changes to the file you didn't intend.  I'm not aware of any way to disassociate photoanalysisd from the library once you've opened it in Photos.
 
-Images used from:
+## Attribution ##
+These tests utilize a test Photos library. The test library is populated with photos from [flickr](https://www.flickr.com) and from my own photo library.  All images used are licensed under Creative Commons 2.0 Attribution [license](https://creativecommons.org/licenses/by/2.0/).  
+
+Flickr images used from:
 - [Jeff Hitchcock](https://www.flickr.com/photos/arbron/48353451872/)
 - [Carlos Montesdeoca](https://www.flickr.com/photos/carlosmontesdeocastudio)
 - [Rydale Clothing](https://www.flickr.com/photos/rydaleclothing)
 - [Marco Verch](https://www.flickr.com/photos/30478819@N08/48228222317/)
 - [K M](https://www.flickr.com/photos/153387643@N08/49334338022/)
-
+- [Shelby Mash](https://www.flickr.com/photos/shelbzyleigh/3809603052)
+- [Rory MacLeod](https://www.flickr.com/photos/macrj/6969547134)
+- [Md. Al Amin](https://www.flickr.com/photos/alamin_bd/45207044465)
+- [Fatlum Haliti](https://www.flickr.com/photos/lumlumi/363449752)
+- [Benny Mazur](https://www.flickr.com/photos/benimoto/399012465)
+- [Sara Cooper PR](https://www.flickr.com/photos/saracooperpr/6422472677)
+- [herval](https://www.flickr.com/photos/herval/2403994289)
+- [Vox Efx](https://www.flickr.com/photos/vox_efx/141137669)
+- [Bill Strain](https://www.flickr.com/photos/billstrain/5117042252)
+- [Guilherme Yagui](https://www.flickr.com/photos/yagui7/15895161088/)
+- [Deborah Austin](https://www.flickr.com/photos/littledebbie11/8703591799/)
+- [We Are Social](https://www.flickr.com/photos/wearesocial/23309711462/)
+- [cloud.shepherd](https://www.flickr.com/photos/exnucboy1/31017877125)
 

tests/Test-10.14.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -36,7 +36,7 @@
 			<key>other</key>
 			<integer>0</integer>
 			<key>photos</key>
-			<integer>6</integer>
+			<integer>11</integer>
 			<key>videos</key>
 			<integer>0</integer>
 		</dict>

tests/Test-10.14.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-04-25T23:54:43Z</date>
+	<date>2020-10-09T16:14:42Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-04-26T06:26:10Z</date>
+	<date>2020-10-10T05:21:03Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/private/com.apple.photomodel/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2020-04-17T17:51:16Z</date>
+		<date>2020-10-04T23:49:39Z</date>
 	</dict>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2020-04-25T23:54:29Z</date>
+	<date>2020-10-04T23:43:17Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>606</integer>
+	<integer>948</integer>
 	<key>LibraryBuildTag</key>
 	<string>D8C4AAA1-3AB6-4A65-BEBD-99CC3E5D433E</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.14.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>606</integer>
+		<integer>948</integer>
 		<key>LibraryBuildTag</key>
 		<string>D8C4AAA1-3AB6-4A65-BEBD-99CC3E5D433E</string>
 		<key>LibrarySchemaVersion</key>
@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-27T13:16:43Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2020-04-25T23:56:35Z</date>
+	<date>2020-10-10T05:22:36Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>