Implemented PhotoInfo.exiftool

fixed some minor findings...

.github/workflows/pythonpackage.yml

@@ -5,11 +5,11 @@ on: [push]
 jobs:
   build:
 
-    runs-on: macOS-10.14
+    runs-on: macOS-latest
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [3.6, 3.7]
+        python-version: [3.8]
 
     steps:
     - uses: actions/checkout@v1
@@ -21,8 +21,8 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install -r requirements.txt
-    - name: Lint with flake8
-      run: |
+   # - name: Lint with flake8
+   #   run: |
        # pip install flake8
         # stop the build if there are Python syntax errors or undefined names
        # flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
@@ -31,4 +31,5 @@ jobs:
     - name: Test with pytest
       run: |
         pip install pytest
+        pip install pytest-mock
         python -m pytest tests/

CHANGELOG.md

@@ -4,6 +4,323 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.28.17](https://github.com/RhetTbull/osxphotos/compare/v0.28.15...v0.28.17)
+
+> 14 May 2020
+
+- Added ExifInfo (Photos 5 only) [`53304d7`](https://github.com/RhetTbull/osxphotos/commit/53304d702317d007056c1d12064503c3ec4ae6f6)
+- Added as_dict to ExifTool [`d1af14d`](https://github.com/RhetTbull/osxphotos/commit/d1af14dbb4d441a62d352123774e51fa3538db97)
+
+#### [v0.28.15](https://github.com/RhetTbull/osxphotos/compare/v0.28.13...v0.28.15)
+
+> 11 May 2020
+
+- fixed some minor findings... [`#127`](https://github.com/RhetTbull/osxphotos/pull/127)
+- added --export-as-hardlink option [`#126`](https://github.com/RhetTbull/osxphotos/pull/126)
+- Added test for folder_names on 10.15.4, closes #119 [`#119`](https://github.com/RhetTbull/osxphotos/issues/119)
+- Refactored photosdb and photoinfo to add SearchInfo and labels [`98b3f63`](https://github.com/RhetTbull/osxphotos/commit/98b3f63a92aa2105f8fa97af992fc6fe2d78b973)
+- Added additional test for --export-as-hardlink [`57315d4`](https://github.com/RhetTbull/osxphotos/commit/57315d44497fde977956f76f667470208f11aa2d)
+- added CHANGELOG.md [`00e1661`](https://github.com/RhetTbull/osxphotos/commit/00e16611fc86c05fb090d036084db9eb42444071)
+
+#### [v0.28.13](https://github.com/RhetTbull/osxphotos/compare/v0.28.10...v0.28.13)
+
+> 2 May 2020
+
+- added --keyword-template [`65674f5`](https://github.com/RhetTbull/osxphotos/commit/65674f57bc174c078e6c47f12ba3aaba87bfa3a4)
+- Fixed bug related to issue #119 [`7af1ccd`](https://github.com/RhetTbull/osxphotos/commit/7af1ccd4ed22ea7f0f86973bfba7f108b6650291)
+- test library updates [`1b6f661`](https://github.com/RhetTbull/osxphotos/commit/1b6f661e6b59c003d3b8cb35226ffb51469be508)
+
+#### [v0.28.10](https://github.com/RhetTbull/osxphotos/compare/v0.28.8...v0.28.10)
+
+> 29 April 2020
+
+- Bug fix for albums in Photos <= 4 to address issue #116 [`a57da23`](https://github.com/RhetTbull/osxphotos/commit/a57da2346b282d731ed41db600bfc5cbeb1a0992)
+- version bump for pypi [`3fe03cd`](https://github.com/RhetTbull/osxphotos/commit/3fe03cd12752c2a7769007b6d934f1efe9f9c4d2)
+
+#### [v0.28.8](https://github.com/RhetTbull/osxphotos/compare/v0.28.7...v0.28.8)
+
+> 28 April 2020
+
+- Fixed implementation of use_albums_as_keywords and use_persons_as_keywords, closes #115 [`#115`](https://github.com/RhetTbull/osxphotos/issues/115)
+- Updated CHANGELOG.md [`072a8d7`](https://github.com/RhetTbull/osxphotos/commit/072a8d795e5e15fa8ca8d8872aecf4cddd7837f7)
+- Update README.md [`5cc98c3`](https://github.com/RhetTbull/osxphotos/commit/5cc98c338bcc19fd05bf293eb3afe24c07c8b380)
+- Updated README.md [`a800711`](https://github.com/RhetTbull/osxphotos/commit/a80071111f810a1d7d6e2d735839e85499091ea4)
+
+#### [v0.28.7](https://github.com/RhetTbull/osxphotos/compare/v0.28.6...v0.28.7)
+
+> 28 April 2020
+
+- Added --album-keyword and --person-keyword to CLI, closes #61 [`#61`](https://github.com/RhetTbull/osxphotos/issues/61)
+- Updated test libraries [`54d5d4b`](https://github.com/RhetTbull/osxphotos/commit/54d5d4b7ba99204f58e723231309ab6e306be28c)
+- Updated CHANGELOG.md [`38137a1`](https://github.com/RhetTbull/osxphotos/commit/38137a1351cdb7ab72393ea03828933dac0b76b0)
+- Updated tests/README.md [`56a0006`](https://github.com/RhetTbull/osxphotos/commit/56a000609f2f08d0f8800fec49cada2980c3bb9d)
+
+#### [v0.28.6](https://github.com/RhetTbull/osxphotos/compare/v0.28.5...v0.28.6)
+
+> 26 April 2020
+
+- Fixed locale bug in templates, closes #113 [`#113`](https://github.com/RhetTbull/osxphotos/issues/113)
+- Updated CHANGELOG.md [`81d4e39`](https://github.com/RhetTbull/osxphotos/commit/81d4e392c39f0fe6f967a447c7d0c970bf224032)
+- Updated test to avoid issue with GitHub workflow [`9be0f84`](https://github.com/RhetTbull/osxphotos/commit/9be0f849b73061d053d30274ff3295b79c88f0b6)
+- Update pythonpackage.yml to remove older pythons [`ccb5f25`](https://github.com/RhetTbull/osxphotos/commit/ccb5f252d14e9335ae04a2e338a6d527b80c9a93)
+
+#### [v0.28.5](https://github.com/RhetTbull/osxphotos/compare/0.28.2...v0.28.5)
+
+> 21 April 2020
+
+- added __len__ to PhotosDB, closes #44 [`#44`](https://github.com/RhetTbull/osxphotos/issues/44)
+- Updated use of _PHOTOS_4_VERSION, closes #106 [`#106`](https://github.com/RhetTbull/osxphotos/issues/106)
+- Updated tests and test library with RAW images [`9b9b54e`](https://github.com/RhetTbull/osxphotos/commit/9b9b54e590e43ae49fb3ae41d493a1f8faec4181)
+- Updated setup.py to resolve issue with bpylist2 on python < 3.8 [`8e4b88a`](https://github.com/RhetTbull/osxphotos/commit/8e4b88ad1fc18438f941e045bfc8aeac878914f9)
+- Added cli.py for use with pyinstaller [`cf28cb6`](https://github.com/RhetTbull/osxphotos/commit/cf28cb6452de17f2ef8d80435386e8d5a1aabd34)
+
+#### [0.28.2](https://github.com/RhetTbull/osxphotos/compare/v0.28.1...0.28.2)
+
+> 18 April 2020
+
+- Added folder support for Photos <= 4, closes #93 [`#93`](https://github.com/RhetTbull/osxphotos/issues/93)
+- cleaned up SQL statements in _process_database4 [`6f28171`](https://github.com/RhetTbull/osxphotos/commit/6f281711e2001a63ffad076d7b9835272d5d09da)
+- Updated CHANGELOG.md [`1fa9583`](https://github.com/RhetTbull/osxphotos/commit/1fa9583ea689d54d2613a064f1ade25bcdfbf043)
+- Fixed suffix check on export to be case insensitive [`4b30b3b`](https://github.com/RhetTbull/osxphotos/commit/4b30b3b4260e2c7409e18825e5b626efe646db16)
+
+#### [v0.28.1](https://github.com/RhetTbull/osxphotos/compare/v0.27.4...v0.28.1)
+
+> 18 April 2020
+
+- Initial work on suppport for associated RAW images [`7e42ebb`](https://github.com/RhetTbull/osxphotos/commit/7e42ebb2402d45cd5d20bdd55bddddaa9db4679f)
+- Initial support for RAW photos in Photos 4 to address issue #101 [`9d15147`](https://github.com/RhetTbull/osxphotos/commit/9d151478d610291b8d482aafae3d445dfd391fca)
+- replaced CLI option --original-name with --current-name [`36c2821`](https://github.com/RhetTbull/osxphotos/commit/36c2821a0fa62eaaa54cf1edc2d9c6da98155354)
+
+#### [v0.27.4](https://github.com/RhetTbull/osxphotos/compare/v0.27.3...v0.27.4)
+
+> 12 April 2020
+
+- Added {folder_album} to template and --folder to CLI [`b7c7b9f`](https://github.com/RhetTbull/osxphotos/commit/b7c7b9f0664e69c743bdd8a228ad2936cf6b7600)
+- Test library update [`21e7020`](https://github.com/RhetTbull/osxphotos/commit/21e7020fec406b0f3926d7adc8a1451bfe77e75a)
+- Updated CHANGELOG.md [`952741d`](https://github.com/RhetTbull/osxphotos/commit/952741d488d2fbbaf8a0c1d3781ad7c4205c068f)
+
+#### [v0.27.3](https://github.com/RhetTbull/osxphotos/compare/v0.27.1...v0.27.3)
+
+> 12 April 2020
+
+- Added additional tests for album_info [`97362fc`](https://github.com/RhetTbull/osxphotos/commit/97362fc0f13b2867abc013f4ba97ae60b0700894)
+- Fixed bug with handling of deleted albums [`9fef12e`](https://github.com/RhetTbull/osxphotos/commit/9fef12ed37634a7bdb11232976b4b2ddccd1a7cb)
+
+#### [v0.27.1](https://github.com/RhetTbull/osxphotos/compare/v0.27.0...v0.27.1)
+
+> 12 April 2020
+
+- Changed AlbumInfo and FolderInfo interface to maintain backwards compatibility with PhotosDB.albums [`e09f0b4`](https://github.com/RhetTbull/osxphotos/commit/e09f0b40f1671d70ee399cdc519492b04fac8adc)
+- Updated CHANGELOG.md [`b749681`](https://github.com/RhetTbull/osxphotos/commit/b749681c6d2545eacf653ab1b2a5d1384e3123eb)
+
+#### [v0.27.0](https://github.com/RhetTbull/osxphotos/compare/v0.26.1...v0.27.0)
+
+> 11 April 2020
+
+- Update README.md [`#95`](https://github.com/RhetTbull/osxphotos/pull/95)
+- Added tests and README for AlbumInfo and FolderInfo [`d6a22b7`](https://github.com/RhetTbull/osxphotos/commit/d6a22b765ab17f6ef1ba8c50b77946f090979968)
+- Added albuminfo.py for AlbumInfo and FolderInfo classes [`9636572`](https://github.com/RhetTbull/osxphotos/commit/96365728c2ff42abfb6828872ffac53b4c3c8024)
+- Updated CHANGELOG.md [`cde56e9`](https://github.com/RhetTbull/osxphotos/commit/cde56e9d13baf3098ec85839cf1aaa33b4915ac9)
+
+#### [v0.26.1](https://github.com/RhetTbull/osxphotos/compare/v0.26.0...v0.26.1)
+
+> 11 April 2020
+
+- Bug fix for PhotosDB.photos() query [`1c9da5e`](https://github.com/RhetTbull/osxphotos/commit/1c9da5ed6ffa21f0577906b65b7da08951725d1f)
+- Updated test library [`d74f7f4`](https://github.com/RhetTbull/osxphotos/commit/d74f7f499bf59f37ec81cfa9d49cbbf3aafb5961)
+- Updated CHANGELOG.md [`c85bb02`](https://github.com/RhetTbull/osxphotos/commit/c85bb023042e072d6688060eb259156c2fa579b9)
+
+#### [v0.26.0](https://github.com/RhetTbull/osxphotos/compare/v0.25.1...v0.26.0)
+
+> 11 April 2020
+
+- Added test for 10.15.4 [`1820715`](https://github.com/RhetTbull/osxphotos/commit/182071584904d001a9b199eef5febfb79e00696e)
+- Changed PhotosDB albums interface as prep for adding folders [`3e50626`](https://github.com/RhetTbull/osxphotos/commit/3e5062684ab6d706d91d4abeb4e3b0ca47867b70)
+- Updated CHANGELOG.md [`a6ca3f4`](https://github.com/RhetTbull/osxphotos/commit/a6ca3f453ce0fae4e8d13c7c256ed69a16d2e3f2)
+
+#### [v0.25.1](https://github.com/RhetTbull/osxphotos/compare/v0.25.0...v0.25.1)
+
+> 5 April 2020
+
+- Added --no-extended-attributes option to CLI, closes #85 [`#85`](https://github.com/RhetTbull/osxphotos/issues/85)
+- Fixed CLI help for invalid topic, closes #76 [`#76`](https://github.com/RhetTbull/osxphotos/issues/76)
+- Updated test library [`bae0283`](https://github.com/RhetTbull/osxphotos/commit/bae0283441f04d71aa78dbd1cf014f376ef1f91a)
+
+#### [v0.25.0](https://github.com/RhetTbull/osxphotos/compare/v0.24.2...v0.25.0)
+
+> 4 April 2020
+
+- Added places, --place, --no-place to CLI, closes #87, #88 [`#87`](https://github.com/RhetTbull/osxphotos/issues/87)
+- Updated render_filepath_template to support multiple values [`6a89888`](https://github.com/RhetTbull/osxphotos/commit/6a898886ddadc9d5bc9dbad6ee7365270dd0a26d)
+- Added {album}, {keyword}, and {person} to template system [`507c4a3`](https://github.com/RhetTbull/osxphotos/commit/507c4a374014f999ca19789bce0df0c14332e021)
+- Added places command to CLI [`fd5e748`](https://github.com/RhetTbull/osxphotos/commit/fd5e748dca759ea1c3a7329d447f363afe8418b7)
+
+#### [v0.24.2](https://github.com/RhetTbull/osxphotos/compare/v0.24.1...v0.24.2)
+
+> 28 March 2020
+
+- added {place.country_code} to template system [`be2e167`](https://github.com/RhetTbull/osxphotos/commit/be2e16769d5d2c75af6d7792f1311f5a65c3bc67)
+
+#### [v0.24.1](https://github.com/RhetTbull/osxphotos/compare/v0.23.4...v0.24.1)
+
+> 28 March 2020
+
+- Added detailed place data in PlaceInfo.names [`c06dd42`](https://github.com/RhetTbull/osxphotos/commit/c06dd4233f917f068c087f5604013d371b0a826a)
+- Template system now supports default values [`67a9a9e`](https://github.com/RhetTbull/osxphotos/commit/67a9a9e21bd05d01a3202b0a1279487f5d04c9d9)
+- Replaced template renderer with regex-based renderer [`427c4c0`](https://github.com/RhetTbull/osxphotos/commit/427c4c0bc49f671477866d30eee74834c67d7bc5)
+
+#### [v0.23.4](https://github.com/RhetTbull/osxphotos/compare/v0.23.3...v0.23.4)
+
+> 22 March 2020
+
+- Added export_by_album.py to examples [`908fead`](https://github.com/RhetTbull/osxphotos/commit/908fead8a2fbcef3b4a387f34d83d88c507c5939)
+- Updated CHANGELOG.md [`072e894`](https://github.com/RhetTbull/osxphotos/commit/072e894e56c4dfe5522d073b202933fed0204ef5)
+- Updated pathvalidate calls [`d066435`](https://github.com/RhetTbull/osxphotos/commit/d066435e3df4062be6a0a3d5fa7308f293e764d5)
+
+#### [v0.23.3](https://github.com/RhetTbull/osxphotos/compare/v0.23.1...v0.23.3)
+
+> 22 March 2020
+
+- Initial version of templating system for CLI [`2feb099`](https://github.com/RhetTbull/osxphotos/commit/2feb0999b3f9ffd9a24e37238f780239a027aa49)
+- Added __str__ to place [`ad58b03`](https://github.com/RhetTbull/osxphotos/commit/ad58b03f2d31daf33849b141570dd0fb5e0a262e)
+- Test library updates [`e90d9c6`](https://github.com/RhetTbull/osxphotos/commit/e90d9c6e11fce7a4e4aa348dcc5f57420c0b6c44)
+
+#### [v0.23.1](https://github.com/RhetTbull/osxphotos/compare/v0.23.0...v0.23.1)
+
+> 21 March 2020
+
+- Fixed requirements.txt for bplist2 [`cda5f44`](https://github.com/RhetTbull/osxphotos/commit/cda5f446933ea2272409d1f153e2a7811626ada6)
+- Updated CHANGELOG.md [`b8da976`](https://github.com/RhetTbull/osxphotos/commit/b8da9765b8949eb90852d249c2877eeb1806d987)
+- Updated requirements.txt [`9da7ad6`](https://github.com/RhetTbull/osxphotos/commit/9da7ad6dcc021fdafe358d74e1c52f69dc49ade8)
+
+#### [v0.23.0](https://github.com/RhetTbull/osxphotos/compare/v0.22.23...v0.23.0)
+
+> 21 March 2020
+
+- Added PhotoInfo.place for reverse geolocation data [`b338b34`](https://github.com/RhetTbull/osxphotos/commit/b338b34d5055a7621e4ebe4fbbae12227d77af6d)
+- Updated CHANGELOG.md [`816b98e`](https://github.com/RhetTbull/osxphotos/commit/816b98e617c30d0bdb51bc2413f9915742c8592e)
+- Update pythonpackage.yml [`92e5bdd`](https://github.com/RhetTbull/osxphotos/commit/92e5bdd2e986e5de2a710abf60ba0dc99c6a6730)
+
+#### [v0.22.23](https://github.com/RhetTbull/osxphotos/compare/v0.22.21...v0.22.23)
+
+> 15 March 2020
+
+- Lots of work on export code [`0940f03`](https://github.com/RhetTbull/osxphotos/commit/0940f039d3e628dc4f25c69bf27ce413807d3f71)
+- test library update [`1e08a74`](https://github.com/RhetTbull/osxphotos/commit/1e08a7449e69965a37373dadabb37c993d93fc69)
+
+#### [v0.22.21](https://github.com/RhetTbull/osxphotos/compare/v0.22.17...v0.22.21)
+
+> 15 March 2020
+
+- Working on export edited bug for issue #78 [`8542e1a`](https://github.com/RhetTbull/osxphotos/commit/8542e1a97f6b640f287b37af9e50fd05f964ec4d)
+- Fixed download-missing to only download when actually missing [`dd20b8d`](https://github.com/RhetTbull/osxphotos/commit/dd20b8d8ac3b16d3b72a26b97dcc620b11e3a7c0)
+- Updated CHANGELOG.md [`cc9220e`](https://github.com/RhetTbull/osxphotos/commit/cc9220e0763816d784f2fd8377dfe14a99981622)
+
+#### [v0.22.17](https://github.com/RhetTbull/osxphotos/compare/v0.22.16...v0.22.17)
+
+> 14 March 2020
+
+- Added MANIFEST.in [`279ab36`](https://github.com/RhetTbull/osxphotos/commit/279ab369295cfe1c778b38e212248271e4fc659e)
+- version bump [`783e097`](https://github.com/RhetTbull/osxphotos/commit/783e097da35a210a2aa5c75865a8599541b9da0b)
+
+#### [v0.22.16](https://github.com/RhetTbull/osxphotos/compare/v0.22.13...v0.22.16)
+
+> 14 March 2020
+
+- removed activate from --download-missing-photos Applescript, closes #69 [`#69`](https://github.com/RhetTbull/osxphotos/issues/69)
+- Added media type specials to json and string output, closes #68 [`#68`](https://github.com/RhetTbull/osxphotos/issues/68)
+- Added query/export options for special media types [`2b7d84a`](https://github.com/RhetTbull/osxphotos/commit/2b7d84a4d103982ad874d875bafbc34d654d539a)
+- README.md update [`a27ce33`](https://github.com/RhetTbull/osxphotos/commit/a27ce33473df3260dfb7ed26e28295cbf87d1e78)
+- Test library updates [`2d7d0b8`](https://github.com/RhetTbull/osxphotos/commit/2d7d0b86e0008cae043e314937504f36ad882990)
+
+#### [v0.22.13](https://github.com/RhetTbull/osxphotos/compare/v0.22.12...v0.22.13)
+
+> 8 March 2020
+
+- Added media type specials, closes #60 [`#60`](https://github.com/RhetTbull/osxphotos/issues/60)
+- Updated CHANGELOG.md [`08a9793`](https://github.com/RhetTbull/osxphotos/commit/08a9793651481e1984a4482794ffedd48e4367a2)
+- Updated README.md [`1f8fd6e`](https://github.com/RhetTbull/osxphotos/commit/1f8fd6e929cc0edd3dd2f222416454d26955bf2a)
+
+#### [v0.22.12](https://github.com/RhetTbull/osxphotos/compare/0.22.10...v0.22.12)
+
+> 7 March 2020
+
+- Added exiftool [`8dea419`](https://github.com/RhetTbull/osxphotos/commit/8dea41961bad285be7058a68e5f7199e5cfb740e)
+- Added --exiftool to CLI export [`ef79961`](https://github.com/RhetTbull/osxphotos/commit/ef799610aea67b703a7d056b7eee227534ba78a5)
+- Updated test library [`9a0fc0d`](https://github.com/RhetTbull/osxphotos/commit/9a0fc0db3e79359610fd0f124a97b03fcf97d8a7)
+
+#### [0.22.10](https://github.com/RhetTbull/osxphotos/compare/v0.22.9...0.22.10)
+
+> 8 February 2020
+
+- Fixed bug in --download-missing to fix issue #64 [`c654e3d`](https://github.com/RhetTbull/osxphotos/commit/c654e3dc61283382b37b6892dab1516ec517143a)
+- removed commented out code [`69addc3`](https://github.com/RhetTbull/osxphotos/commit/69addc34649f992c6a4a0e0e334754a72530f0ba)
+- Updated CHANGELOG.md [`1e013b6`](https://github.com/RhetTbull/osxphotos/commit/1e013b6802e49e26ec5a94eb702e841b2eb68395)
+
+#### [v0.22.9](https://github.com/RhetTbull/osxphotos/compare/v0.22.7...v0.22.9)
+
+> 1 February 2020
+
+- Updated PhotosDB to only copy database if locked, speed improvement for cases where DB not locked; closes #34 [`#34`](https://github.com/RhetTbull/osxphotos/issues/34)
+- Changed temp file handling to use tempfile.TemporaryDirectory, closes #59 [`#59`](https://github.com/RhetTbull/osxphotos/issues/59)
+- Slight refactor to PhotosDB.photos() [`91d5729`](https://github.com/RhetTbull/osxphotos/commit/91d5729beaa0f0c2583e6320b18d958429e66075)
+- Test library updates [`6e563e2`](https://github.com/RhetTbull/osxphotos/commit/6e563e214c569ba7838f7464de9258c3bba5db23)
+- Removed _tmp_file code that's no longer needed [`27994c9`](https://github.com/RhetTbull/osxphotos/commit/27994c9fd372303833a5794f1de9815f425c762e)
+
+#### [v0.22.7](https://github.com/RhetTbull/osxphotos/compare/v0.22.4...v0.22.7)
+
+> 27 January 2020
+
+- Corrected Panorama Flag [`#58`](https://github.com/RhetTbull/osxphotos/pull/58)
+- Jan 20 Updates [`#1`](https://github.com/RhetTbull/osxphotos/pull/1)
+- Added XMP sidecar option to export, closes #51 [`#51`](https://github.com/RhetTbull/osxphotos/issues/51)
+- Test library updates, closes #52 [`#52`](https://github.com/RhetTbull/osxphotos/issues/52)
+- Added XMP sidecar to export [`4dfb131`](https://github.com/RhetTbull/osxphotos/commit/4dfb131a21b1b1efefe3b918ecb06fc6fcb03f2c)
+- Added date_modified to PhotoInfo [`67b0ae0`](https://github.com/RhetTbull/osxphotos/commit/67b0ae0bf679815372d415c3064e21d46a5b8718)
+- Added date_modified to PhotoInfo [`4d36b3b`](https://github.com/RhetTbull/osxphotos/commit/4d36b3b31f3e0e74d9d111b6b691771e19f94086)
+
+#### [v0.22.4](https://github.com/RhetTbull/osxphotos/compare/v0.22.0...v0.22.4)
+
+> 20 January 2020
+
+- Add --from-date and --to-date to query and export command [`#57`](https://github.com/RhetTbull/osxphotos/pull/57)
+- Refactor CLI [`#55`](https://github.com/RhetTbull/osxphotos/pull/55)
+- Refactor cli: singular --db, --json and query options. [`e214746`](https://github.com/RhetTbull/osxphotos/commit/e214746063271e6f9f586286103ed051ada49d85)
+- Implement from_date and to_date in PhotosDB as well as query and export command. Some refactoring of CLI as well. [`cfa2b4a`](https://github.com/RhetTbull/osxphotos/commit/cfa2b4a828facf0aff5bc19f777457ad776c4a05)
+- Refactored _query.  Still hairy, but less so. [`b9dee49`](https://github.com/RhetTbull/osxphotos/commit/b9dee4995c6d89fadb3d2482374b7098f2ab5ed9)
+
+#### [v0.22.0](https://github.com/RhetTbull/osxphotos/compare/v0.21.5...v0.22.0)
+
+> 18 January 2020
+
+- Refactored PhotosDB and CLI to require explicity passing the database to avoid non-deterministic behavior when last database can't be found.  This may break existing code. [`ede56ff`](https://github.com/RhetTbull/osxphotos/commit/ede56ffc31cf98811b3d4d16e22406ac0eae0315)
+- Changed get_system_library_path to return None if could not get system library [`646ea4f`](https://github.com/RhetTbull/osxphotos/commit/646ea4f24ca1119b27280af1445e31adcd0690f0)
+- Updated CHANGELOG.md [`bd20388`](https://github.com/RhetTbull/osxphotos/commit/bd20388778dfa645277029601c63fc9835b7a406)
+
+#### [v0.21.5](https://github.com/RhetTbull/osxphotos/compare/v0.21.0...v0.21.5)
+
+> 13 January 2020
+
+- Fixed search for edited photo in path_edited [`edb31f7`](https://github.com/RhetTbull/osxphotos/commit/edb31f796a76912e6ed8182b691396cf4ec62ffa)
+- Added tests for live photos [`5473f3b`](https://github.com/RhetTbull/osxphotos/commit/5473f3b3fd745d4772721dfd1ed821ab0660bf72)
+- Added incloud and iscloudasset for Photos 4 [`e089d13`](https://github.com/RhetTbull/osxphotos/commit/e089d135d3e04320bf98b2c9b11875343e68be04)
+
+#### [v0.21.0](https://github.com/RhetTbull/osxphotos/compare/v0.20.0...v0.21.0)
+
+> 4 January 2020
+
+- Added live photo support for both Photos 4 & 5 [`d5eaff0`](https://github.com/RhetTbull/osxphotos/commit/d5eaff02f2a29a9d105ab72e9a9aeffbc9a3425b)
+- Added support for burst photos; added export-bursts to CLI [`593983a`](https://github.com/RhetTbull/osxphotos/commit/593983a09940e67fb9347bf345cfd7289465fa0a)
+- Added live-photo option to CLI query and export [`6f6d37c`](https://github.com/RhetTbull/osxphotos/commit/6f6d37ceacf71a52a2c0216f0ad75afee244946a)
+
+#### [v0.20.0](https://github.com/RhetTbull/osxphotos/compare/v0.19.0...v0.20.0)
+
+> 1 January 2020
+
+- Added support for filtering only movies or photos to CLI; added search for UTI to CLI [`9cd5363`](https://github.com/RhetTbull/osxphotos/commit/9cd5363a800dd85f333219788c661745b2ce88ad)
+- Added support for bust photos; added export-bursts to CLI [`1136f84`](https://github.com/RhetTbull/osxphotos/commit/1136f84d9b5ea454115ba3d2720625722671e63b)
+- Temporary fix to filter out unselected burst photos [`a550ba0`](https://github.com/RhetTbull/osxphotos/commit/a550ba00d6ff43a819cb18446e532f10ded81834)
+
 #### [v0.19.0](https://github.com/RhetTbull/osxphotos/compare/v0.18.0...v0.19.0)
 
 > 29 December 2019
@@ -22,7 +339,11 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Added test cases and documentation for shared photos and shared albums [`6d20e9e`](https://github.com/RhetTbull/osxphotos/commit/6d20e9e36185aa027d82237cadfe3b55614ba96f)
 - Refactored PhotoInfo to use properties instead of methods--major update [`1ddd90c`](https://github.com/RhetTbull/osxphotos/commit/1ddd90cbdc824afc5df9d2347e730bd9f86350ee)
 
-#### [v0.15.1](https://github.com/RhetTbull/osxphotos/compare/v0.14.21...v0.15.1)
+#### [v0.15.1](https://github.com/RhetTbull/osxphotos/compare/v0.15.0...v0.15.1)
+
+> 14 May 2020
+
+#### [v0.15.0](https://github.com/RhetTbull/osxphotos/compare/v0.14.21...v0.15.0)
 
 > 14 December 2019
 

MANIFEST.in

@@ -0,0 +1,2 @@
+include README.md
+include osxphotos/templates/*

cli.py

@@ -0,0 +1,19 @@
+""" stand alone command line script for use with pyinstaller
+    
+    To build this into an executable:
+    - install pyinstaller:
+        python3 -m pip install pyinstaller
+    - then use make_cli_exe.sh to run pyinstaller or execute the following command:
+        pyinstaller --onefile --hidden-import="pkg_resources.py2_warn" --name osxphotos --add-data osxphotos/templates/xmp_sidecar.mako:osxphotos/templates cli.py
+
+    Resulting executable will be in "dist/osxphotos"
+
+    Note: This is *not* the cli that "python3 -m pip install osxphotos" or "python setup.py install" would install;
+    it's merely a wrapper around __main__.py to allow pyinstaller to work
+    
+"""
+
+from osxphotos.__main__ import cli
+
+if __name__ == "__main__":
+    cli()

examples/examples.py

@@ -1,8 +1,14 @@
 import osxphotos
+import os.path
 
 
 def main():
-    photosdb = osxphotos.PhotosDB()
+    db = osxphotos.utils.get_system_library_path()
+    if db is None:
+        # Note: get_system_library_path only works on MacOS 10.15+
+        db = os.path.expanduser("~/Pictures/Photos Library.photoslibrary")
+
+    photosdb = osxphotos.PhotosDB(db)
     print(f"db file = {photosdb.db_path}")
     print(f"db version = {photosdb.db_version}")
 

examples/export.py

@@ -8,7 +8,8 @@ import osxphotos
 
 
 def main():
-    photosdb = osxphotos.PhotosDB()
+    db = os.path.expanduser("~/Pictures/Photos Library.photoslibrary")
+    photosdb = osxphotos.PhotosDB(db)
     photos = photosdb.photos()
 
     export_path = os.path.expanduser("~/Desktop/export")

examples/export_by_album.py

@@ -0,0 +1,86 @@
+""" Export all photos to specified directory using album names as folders
+    If file has been edited, also export the edited version, 
+    otherwise, export the original version 
+    This will result in duplicate photos if photo is in more than album """
+
+import os.path
+import pathlib
+import sys
+
+import click
+from pathvalidate import is_valid_filepath, sanitize_filepath
+
+import osxphotos
+
+
+@click.command()
+@click.argument("export_path", type=click.Path(exists=True))
+@click.option(
+    "--default-album",
+    help="Default folder for photos with no album. Defaults to 'unfiled'",
+    default="unfiled",
+)
+@click.option(
+    "--library-path",
+    help="Path to Photos library, default to last used library",
+    default=None,
+)
+@click.option(
+    "--edited",
+    help="Also export edited versions of photos (default is originals only)",
+    is_flag=True,
+    default=False,
+)
+def export(export_path, default_album, library_path, edited):
+    """ Export all photos, organized by album """
+    export_path = os.path.expanduser(export_path)
+    library_path = os.path.expanduser(library_path) if library_path else None
+
+    if library_path is not None:
+        photosdb = osxphotos.PhotosDB(library_path)
+    else:
+        photosdb = osxphotos.PhotosDB()
+
+    photos = photosdb.photos()
+
+    for p in photos:
+        if not p.ismissing:
+            albums = p.albums
+            if not albums:
+                albums = [default_album]
+            for album in albums:
+                click.echo(f"exporting {p.original_filename} in album {album}")
+
+                # make sure no invalid characters in destination path (could be in album name)
+                album_name = sanitize_filepath(album, platform="auto")
+
+                # create destination folder, if necessary, based on album name
+                dest_dir = os.path.join(export_path, album_name)
+
+                # verify path is a valid path
+                if not is_valid_filepath(dest_dir, platform="auto"):
+                    sys.exit(f"Invalid filepath {dest_dir}")
+
+                # create destination dir if needed
+                if not os.path.isdir(dest_dir):
+                    os.makedirs(dest_dir)
+
+                filename = p.original_filename
+                # export the photo but only if --edited, photo has adjustments, and 
+                # path_edited is not None (can be None if edited photo is missing)
+                if edited and p.hasadjustments and p.path_edited:
+                    # export edited version
+                    # use original filename with _edited appended but make sure suffix is 
+                    # same as edited file
+                    edited_filename = f"{pathlib.Path(filename).stem}_edited{pathlib.Path(p.path_edited).suffix}"
+                    exported = p.export(dest_dir, edited_filename, edited=True)
+                    click.echo(f"Exported {edited_filename} to {exported}")
+                # export unedited version
+                exported = p.export(dest_dir, filename)
+                click.echo(f"Exported {filename} to {exported}")
+        else:
+            click.echo(f"Skipping missing photo: {p.original_filename} in album {album}")
+
+
+if __name__ == "__main__":
+    export()  # pylint: disable=no-value-for-parameter

examples/photos_repl.py

@@ -0,0 +1,48 @@
+#!/usr/bin/env python3 -i
+
+# Open an interactive REPL with photosdb and photos defined
+# as osxphotos.PhotosDB() and PhotosDB.photos respectively
+# useful for debugging or exploring the Photos database
+
+# If you run this using python from command line, do so with -i flag:
+# python3 -i examples/photos_repl.py
+
+import sys
+import time
+
+# click needed since this uses a couple of functions from CLI (__main__.py)
+import click
+
+import osxphotos
+from osxphotos.__main__ import get_photos_db, _list_libraries
+
+
+def main():
+    db = None
+
+    if len(sys.argv) > 1:
+        db = sys.argv[1]
+    else:
+        db = get_photos_db()
+
+    if db:
+        print("loading database")
+        tic = time.perf_counter()
+        photosdb = osxphotos.PhotosDB(dbfile=db)
+        toc = time.perf_counter()
+        print(f"done: took {toc-tic} seconds")
+        return photosdb
+    else:
+        _list_libraries()
+        sys.exit()
+
+
+if __name__ == "__main__":
+    print(f"osxphotos version: {osxphotos._version.__version__}")
+    photosdb = main()
+    print(f"database version: {photosdb.db_version}")
+    print("getting photos")
+    tic = time.perf_counter()
+    photos = photosdb.photos(images=True, movies=True)
+    toc = time.perf_counter()
+    print(f"found {len(photos)} photos in {toc-tic} seconds")

make_cli_exe.sh

@@ -0,0 +1,8 @@
+#!/bin/sh 
+
+# This will build an stand-alone executable called 'osxphotos' in your ./dist directory
+# using pyinstaller
+# If you need to install pyinstaller:
+# python3 -m pip install --upgrade pyinstaller
+
+pyinstaller --onefile --hidden-import="pkg_resources.py2_warn" --name osxphotos --add-data osxphotos/templates/xmp_sidecar.mako:osxphotos/templates cli.py

osxphotos/_applescript/__init__.py

@@ -0,0 +1,162 @@
+""" applescript -- Easy-to-use Python wrapper for NSAppleScript """
+
+import sys
+
+from Foundation import NSAppleScript, NSAppleEventDescriptor, NSURL, \
+		NSAppleScriptErrorMessage, NSAppleScriptErrorBriefMessage, \
+		NSAppleScriptErrorNumber, NSAppleScriptErrorAppName, NSAppleScriptErrorRange
+
+from .aecodecs import Codecs, fourcharcode, AEType, AEEnum
+from . import kae
+
+__all__ = ['AppleScript', 'ScriptError', 'AEType', 'AEEnum', 'kMissingValue', 'kae']
+
+
+######################################################################
+
+
+class AppleScript:
+	""" Represents a compiled AppleScript. The script object is persistent; its handlers may be called multiple times and its top-level properties will retain current state until the script object's disposal.
+	
+	
+	"""
+
+	_codecs = Codecs()
+	
+	def __init__(self, source=None, path=None):
+		"""
+			source : str | None -- AppleScript source code
+			path : str | None -- full path to .scpt/.applescript file
+			
+			Notes:
+			
+			- Either the path or the source argument must be provided.
+			
+			- If the script cannot be read/compiled, a ScriptError is raised.
+		"""
+		if path:
+			url = NSURL.fileURLWithPath_(path)
+			self._script, errorinfo = NSAppleScript.alloc().initWithContentsOfURL_error_(url, None)
+			if errorinfo:
+				raise ScriptError(errorinfo)
+		elif source:
+			self._script = NSAppleScript.alloc().initWithSource_(source)
+		else:
+			raise ValueError("Missing source or path argument.")
+		if not self._script.isCompiled():
+			errorinfo = self._script.compileAndReturnError_(None)[1]
+			if errorinfo:
+				raise ScriptError(errorinfo)
+	
+	def __repr__(self):
+		s = self.source
+		return 'AppleScript({})'.format(repr(s) if len(s) < 100 else '{}...{}'.format(repr(s)[:80], repr(s)[-17:]))
+	
+	##
+	
+	def _newevent(self, suite, code, args):
+		evt = NSAppleEventDescriptor.appleEventWithEventClass_eventID_targetDescriptor_returnID_transactionID_(
+						fourcharcode(suite), fourcharcode(code), NSAppleEventDescriptor.nullDescriptor(), 0, 0)
+		evt.setDescriptor_forKeyword_(self._codecs.pack(args), fourcharcode(kae.keyDirectObject))
+		return evt
+	
+	def _unpackresult(self, result, errorinfo):
+		if not result:
+			raise ScriptError(errorinfo)
+		return self._codecs.unpack(result)
+
+	##
+	
+	source = property(lambda self: str(self._script.source()), doc="str -- the script's source code")
+	
+	def run(self, *args):
+		""" Run the script, optionally passing arguments to its run handler.
+			
+				args : anything -- arguments to pass to script, if any; see also supported type mappings documentation
+				Result : anything | None -- the script's return value, if any
+			
+			Notes:
+			
+			- The run handler must be explicitly declared in order to pass arguments.
+			
+			- AppleScript will ignore excess arguments. Passing insufficient arguments will result in an error.
+			
+			- If execution fails, a ScriptError is raised.
+		"""
+		if args:
+			evt = self._newevent(kae.kCoreEventClass, kae.kAEOpenApplication, args)
+			return self._unpackresult(*self._script.executeAppleEvent_error_(evt, None))
+		else:
+			return self._unpackresult(*self._script.executeAndReturnError_(None))
+	
+	def call(self, name, *args):
+		""" Call the specified user-defined handler.
+				
+				name : str -- the handler's name (case-sensitive)
+				args : anything -- arguments to pass to script, if any; see documentation for supported types
+				Result : anything | None -- the script's return value, if any
+			
+			Notes:
+			
+			- The handler's name must be a user-defined identifier, not an AppleScript keyword; e.g. 'myCount' is acceptable; 'count' is not.
+			
+			- AppleScript will ignore excess arguments. Passing insufficient arguments will result in an error.
+			
+			- If execution fails, a ScriptError is raised.
+		"""
+		evt = self._newevent(kae.kASAppleScriptSuite, kae.kASPrepositionalSubroutine, args)
+		evt.setDescriptor_forKeyword_(NSAppleEventDescriptor.descriptorWithString_(name), 
+						fourcharcode(kae.keyASSubroutineName))
+		return self._unpackresult(*self._script.executeAppleEvent_error_(evt, None))
+
+
+##
+
+
+class ScriptError(Exception):
+	""" Indicates an AppleScript compilation/execution error. """
+	
+	def __init__(self, errorinfo):
+		self._errorinfo = dict(errorinfo)
+	
+	def __repr__(self):
+		return 'ScriptError({})'.format(self._errorinfo)
+	
+	@property
+	def message(self):
+		""" str -- the error message """
+		msg = self._errorinfo.get(NSAppleScriptErrorMessage)
+		if not msg:
+			msg = self._errorinfo.get(NSAppleScriptErrorBriefMessage, 'Script Error')
+		return msg
+	
+	number = property(lambda self: self._errorinfo.get(NSAppleScriptErrorNumber),
+			doc="int | None -- the error number, if given")
+	
+	appname = property(lambda self: self._errorinfo.get(NSAppleScriptErrorAppName),
+			doc="str | None -- the name of the application that reported the error, where relevant")
+	
+	@property
+	def range(self):
+		""" (int, int) -- the start and end points (1-indexed) within the source code where the error occurred """
+		range = self._errorinfo.get(NSAppleScriptErrorRange)
+		if range:
+			start = range.rangeValue().location
+			end = start + range.rangeValue().length
+			return (start, end)
+		else:
+			return None
+	
+	def __str__(self):
+		msg = self.message
+		for s, v in [(' ({})', self.number), (' app={!r}', self.appname), (' range={0[0]}-{0[1]}', self.range)]:
+			if v is not None:
+				msg += s.format(v)
+		return msg.encode('ascii', 'replace') if sys.version_info.major < 3 else msg # 2.7 compatibility
+
+
+##
+
+
+kMissingValue = AEType(kae.cMissingValue) # convenience constant
+

osxphotos/_applescript/aecodecs.py

@@ -0,0 +1,269 @@
+""" aecodecs -- Convert from common Python types to Apple Event Manager types and vice-versa. """
+
+import datetime, struct, sys
+
+from Foundation import NSAppleEventDescriptor, NSURL
+
+from . import kae
+
+
+__all__ = ['Codecs', 'AEType', 'AEEnum']
+
+
+######################################################################
+
+
+def fourcharcode(code):
+	""" Convert four-char code for use in NSAppleEventDescriptor methods.
+		
+		code : bytes -- four-char code, e.g. b'utxt'
+		Result : int -- OSType, e.g. 1970567284
+	"""
+	return struct.unpack('>I', code)[0]
+
+
+#######
+
+
+class Codecs:
+	""" Implements mappings for common Python types with direct AppleScript equivalents. Used by AppleScript class. """
+
+	kMacEpoch = datetime.datetime(1904, 1, 1)
+	kUSRF = fourcharcode(kae.keyASUserRecordFields)
+
+	def __init__(self):
+		# Clients may add/remove/replace encoder and decoder items:
+		self.encoders = {
+				NSAppleEventDescriptor.class__(): self.packdesc,
+				type(None): self.packnone,
+				bool: self.packbool,
+				int: self.packint,
+				float: self.packfloat,
+				bytes: self.packbytes,
+				str: self.packstr,
+				list: self.packlist,
+				tuple: self.packlist,
+				dict: self.packdict,
+				datetime.datetime: self.packdatetime,
+				AEType: self.packtype,
+				AEEnum: self.packenum,
+		}
+		if sys.version_info.major < 3: # 2.7 compatibility
+			self.encoders[unicode] = self.packstr
+		
+		self.decoders = {fourcharcode(k): v for k, v in {
+				kae.typeNull: self.unpacknull,
+				kae.typeBoolean: self.unpackboolean,
+				kae.typeFalse: self.unpackboolean,
+				kae.typeTrue: self.unpackboolean,
+				kae.typeSInt32: self.unpacksint32,
+				kae.typeIEEE64BitFloatingPoint: self.unpackfloat64,
+				kae.typeUTF8Text: self.unpackunicodetext,
+				kae.typeUTF16ExternalRepresentation: self.unpackunicodetext,
+				kae.typeUnicodeText: self.unpackunicodetext,
+				kae.typeLongDateTime: self.unpacklongdatetime,
+				kae.typeAEList: self.unpackaelist,
+				kae.typeAERecord: self.unpackaerecord,
+				kae.typeAlias: self.unpackfile,
+				kae.typeFSS: self.unpackfile,
+				kae.typeFSRef: self.unpackfile,
+				kae.typeFileURL: self.unpackfile,
+				kae.typeType: self.unpacktype,
+				kae.typeEnumeration: self.unpackenumeration,
+		}.items()}
+	
+	def pack(self, data):
+		"""Pack Python data.
+			data : anything -- a Python value
+			Result : NSAppleEventDescriptor -- an AE descriptor, or error if no encoder exists for this type of data
+		"""
+		try:
+			return self.encoders[data.__class__](data) # quick lookup by type/class
+		except (KeyError, AttributeError) as e:
+			for type, encoder in self.encoders.items(): # slower but more thorough lookup that can handle subtypes/subclasses
+				if isinstance(data, type):
+					return encoder(data)
+		raise TypeError("Can't pack data into an AEDesc (unsupported type): {!r}".format(data))
+	
+	def unpack(self, desc):
+		"""Unpack an Apple event descriptor.
+			desc : NSAppleEventDescriptor
+			Result : anything -- a Python value, or the original NSAppleEventDescriptor if no decoder is found
+		"""
+		decoder = self.decoders.get(desc.descriptorType())
+		# unpack known type
+		if decoder:
+			return decoder(desc)
+		# if it's a record-like desc, unpack as dict with an extra AEType(b'pcls') key containing the desc type
+		rec = desc.coerceToDescriptorType_(fourcharcode(kae.typeAERecord))
+		if rec:
+			rec = self.unpackaerecord(rec)
+			rec[AEType(kae.pClass)] = AEType(struct.pack('>I', desc.descriptorType()))
+			return rec
+		# return as-is
+		return desc
+	
+	##
+	
+	def _packbytes(self, desctype, data):
+		return NSAppleEventDescriptor.descriptorWithDescriptorType_bytes_length_(
+			fourcharcode(desctype), data, len(data))
+	
+	def packdesc(self, val):
+		return val
+	
+	def packnone(self, val):
+		return NSAppleEventDescriptor.nullDescriptor()
+	
+	def packbool(self, val):
+		return NSAppleEventDescriptor.descriptorWithBoolean_(int(val))
+	
+	def packint(self, val):
+		if (-2**31) <= val < (2**31):
+			return NSAppleEventDescriptor.descriptorWithInt32_(val)
+		else:
+			return self.pack(float(val))
+	
+	def packfloat(self, val):
+		return self._packbytes(kae.typeFloat, struct.pack('d', val))
+	
+	def packbytes(self, val):
+		return self._packbytes(kae.typeData, val)
+	
+	def packstr(self, val):
+		return NSAppleEventDescriptor.descriptorWithString_(val)
+	
+	def packdatetime(self, val):
+		delta = val - self.kMacEpoch
+		sec = delta.days * 3600 * 24 + delta.seconds
+		return self._packbytes(kae.typeLongDateTime, struct.pack('q', sec))
+	
+	def packlist(self, val):
+		lst = NSAppleEventDescriptor.listDescriptor()
+		for item in val:
+			lst.insertDescriptor_atIndex_(self.pack(item), 0)
+		return lst
+	
+	def packdict(self, val):
+		record = NSAppleEventDescriptor.recordDescriptor()
+		usrf = desctype = None
+		for key, value in val.items():
+			if isinstance(key, AEType):
+				if key.code == kae.pClass and isinstance(value, AEType): # AS packs records that contain a 'class' property by coercing the packed record to the descriptor type specified by the property's value (assuming it's an AEType)
+					desctype = value
+				else:
+					record.setDescriptor_forKeyword_(self.pack(value), fourcharcode(key.code))
+			else:
+				if not usrf:
+					usrf = NSAppleEventDescriptor.listDescriptor()
+				usrf.insertDescriptor_atIndex_(self.pack(key), 0)
+				usrf.insertDescriptor_atIndex_(self.pack(value), 0)
+		if usrf:
+			record.setDescriptor_forKeyword_(usrf, self.kUSRF)
+		if desctype:
+			newrecord = record.coerceToDescriptorType_(fourcharcode(desctype.code))
+			if newrecord:
+				record = newrecord
+			else: # coercion failed for some reason, so pack as normal key-value pair
+				record.setDescriptor_forKeyword_(self.pack(desctype), fourcharcode(key.code))
+		return record
+	
+	def packtype(self, val):
+		return NSAppleEventDescriptor.descriptorWithTypeCode_(fourcharcode(val.code))
+	
+	def packenum(self, val): 
+		return NSAppleEventDescriptor.descriptorWithEnumCode_(fourcharcode(val.code))
+	
+	#######
+	
+	def unpacknull(self, desc):
+		return None
+	
+	def unpackboolean(self, desc):
+		return desc.booleanValue()
+	
+	def unpacksint32(self, desc):
+		return desc.int32Value()
+	
+	def unpackfloat64(self, desc):
+		return struct.unpack('d', bytes(desc.data()))[0]
+	
+	def unpackunicodetext(self, desc):
+		return desc.stringValue()
+	
+	def unpacklongdatetime(self, desc):
+		return self.kMacEpoch + datetime.timedelta(seconds=struct.unpack('q', bytes(desc.data()))[0])
+	
+	def unpackaelist(self, desc):
+		return [self.unpack(desc.descriptorAtIndex_(i + 1)) for i in range(desc.numberOfItems())]
+	
+	def unpackaerecord(self, desc):
+		dct = {}
+		for i in range(desc.numberOfItems()):
+			key = desc.keywordForDescriptorAtIndex_(i + 1)
+			value = desc.descriptorForKeyword_(key)
+			if key == self.kUSRF:
+				lst = self.unpackaelist(value)
+				for i in range(0, len(lst), 2):
+					dct[lst[i]] = lst[i+1]
+			else:
+				dct[AEType(struct.pack('>I', key))] = self.unpack(value)
+		return dct
+	
+	def unpacktype(self, desc):
+		return AEType(struct.pack('>I', desc.typeCodeValue()))
+	
+	def unpackenumeration(self, desc):
+		return AEEnum(struct.pack('>I', desc.enumCodeValue()))
+	
+	def unpackfile(self, desc):
+		url = bytes(desc.coerceToDescriptorType_(fourcharcode(kae.typeFileURL)).data()).decode('utf8')
+		return NSURL.URLWithString_(url).path()
+
+
+#######
+
+
+class AETypeBase:
+	""" Base class for AEType and AEEnum.
+	
+		Notes:
+		
+		- Hashable and comparable, so may be used as keys in dictionaries that map to AE records.
+	"""
+	
+	def __init__(self, code):
+		"""
+			code : bytes -- four-char code, e.g. b'utxt'
+		"""
+		if not isinstance(code, bytes):
+			raise TypeError('invalid code (not a bytes object): {!r}'.format(code))
+		elif len(code) != 4:
+			raise ValueError('invalid code (not four bytes long): {!r}'.format(code))
+		self._code = code
+	
+	code = property(lambda self:self._code, doc="bytes -- four-char code, e.g. b'utxt'")
+	
+	def __hash__(self): 
+		return hash(self._code)
+	
+	def __eq__(self, val):
+		return val.__class__ == self.__class__ and val.code == self._code
+	
+	def __ne__(self, val):
+		return not self == val
+		
+	def __repr__(self):
+		return "{}({!r})".format(self.__class__.__name__, self._code)
+
+
+##
+
+
+class AEType(AETypeBase):
+	"""An AE type. Maps to an AppleScript type class, e.g. AEType(b'utxt') <=> 'unicode text'."""
+
+
+class AEEnum(AETypeBase):
+	"""An AE enumeration. Maps to an AppleScript constant, e.g. AEEnum(b'yes ') <=> 'yes'."""
+

osxphotos/_constants.py

@@ -2,6 +2,7 @@
 Constants used by osxphotos 
 """
 
+import os.path
 
 # which Photos library database versions have been tested
 # Photos 2.0 (10.12.6) == 2622
@@ -12,8 +13,12 @@ Constants used by osxphotos
 # TODO: Should this also use compatibleBackToVersion from LiGlobals?
 _TESTED_DB_VERSIONS = ["6000", "4025", "4016", "3301", "2622"]
 
-# versions later than this have a different database structure
-_PHOTOS_5_VERSION = "6000"
+# only version 3 - 4 have RKVersion.selfPortrait
+_PHOTOS_3_VERSION = "3301"
+
+# versions 5.0 and later have a different database structure
+_PHOTOS_4_VERSION = "4025"  # latest Mojove version on 10.14.6
+_PHOTOS_5_VERSION = "6000"  # seems to be current on 10.15.1 through 10.15.4
 
 # which major version operating systems have been tested
 _TESTED_OS_VERSIONS = ["12", "13", "14", "15"]
@@ -21,6 +26,9 @@ _TESTED_OS_VERSIONS = ["12", "13", "14", "15"]
 # Photos 5 has persons who are empty string if unidentified face
 _UNKNOWN_PERSON = "_UNKNOWN_"
 
+# photos with no reverse geolocation info (place)
+_UNKNOWN_PLACE = "_UNKNOWN_"
+
 _EXIF_TOOL_URL = "https://exiftool.org/"
 
 # Where are shared iCloud photos located?
@@ -30,3 +38,28 @@ _PHOTOS_5_SHARED_PHOTO_PATH = "resources/cloudsharing/data"
 _PHOTO_TYPE = 0
 _MOVIE_TYPE = 1
 
+# Name of XMP template file
+_TEMPLATE_DIR = os.path.join(os.path.dirname(__file__), "templates")
+_XMP_TEMPLATE_NAME = "xmp_sidecar.mako"
+
+# Constants used for processing folders and albums
+_PHOTOS_5_ALBUM_KIND = 2  # normal user album
+_PHOTOS_5_SHARED_ALBUM_KIND = 1505  # shared album
+_PHOTOS_5_FOLDER_KIND = 4000  # user folder
+_PHOTOS_5_ROOT_FOLDER_KIND = 3999  # root folder
+
+_PHOTOS_4_ALBUM_KIND = 3  # RKAlbum.albumSubclass
+_PHOTOS_4_TOP_LEVEL_ALBUM = "TopLevelAlbums"
+_PHOTOS_4_ROOT_FOLDER = "LibraryFolder"
+
+# EXIF related constants
+# max keyword length for IPTC:Keyword, reference
+# https://www.iptc.org/std/photometadata/documentation/userguide/
+_MAX_IPTC_KEYWORD_LEN = 64
+
+# Sentinel value for detecting if a template in keyword_template doesn't match
+# If anyone has a keyword matching this, then too bad...
+_OSXPHOTOS_NONE_SENTINEL = "OSXPhotosXYZZY42_Sentinel$"
+
+# SearchInfo categories for Photos 5, corresponds to categories in database/search/psi.sqlite
+SEARCH_CATEGORY_LABEL = 2024

osxphotos/_version.py

@@ -1,3 +1,3 @@
 """ version info """
 
-__version__ = "0.20.00"
+__version__ = "0.28.18"

osxphotos/albuminfo.py

@@ -0,0 +1,211 @@
+"""
+AlbumInfo and FolderInfo classes for dealing with albums and folders
+
+AlbumInfo class
+Represents a single Album in the Photos library and provides access to the album's attributes
+PhotosDB.albums() returns a list of AlbumInfo objects
+
+FolderInfo class
+Represents a single Folder in the Photos library and provides access to the folders attributes
+PhotosDB.folders() returns a list of FolderInfo objects
+"""
+
+import logging
+
+from ._constants import (
+    _PHOTOS_4_ALBUM_KIND,
+    _PHOTOS_4_TOP_LEVEL_ALBUM,
+    _PHOTOS_4_VERSION,
+    _PHOTOS_5_ALBUM_KIND,
+    _PHOTOS_5_FOLDER_KIND,
+)
+
+
+class AlbumInfo:
+    """
+    Info about a specific Album, contains all the details about the album
+    including folders, photos, etc.
+    """
+
+    def __init__(self, db=None, uuid=None):
+        self._uuid = uuid
+        self._db = db
+        self._title = self._db._dbalbum_details[uuid]["title"]
+
+    @property
+    def title(self):
+        """ return title / name of album """
+        return self._title
+
+    @property
+    def uuid(self):
+        """ return uuid of album """
+        return self._uuid
+
+    @property
+    def photos(self):
+        """ return list of photos contained in album """
+        try:
+            return self._photos
+        except AttributeError:
+            uuid = self._db._dbalbums_album[self._uuid]
+            self._photos = self._db.photos(uuid=uuid)
+            return self._photos
+
+    @property
+    def folder_names(self):
+        """ return hierarchical list of folders the album is contained in
+            the folder list is in form:
+            ["Top level folder", "sub folder 1", "sub folder 2", ...]
+            returns empty list if album is not in any folders """
+
+        try:
+            return self._folder_names
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                self._folder_names = self._db._album_folder_hierarchy_list(self._uuid)
+            else:
+                self._folder_names = self._db._album_folder_hierarchy_list(self._uuid)
+            return self._folder_names
+
+    @property
+    def folder_list(self):
+        """ return hierarchical list of folders the album is contained in
+            as list of FolderInfo objects in form 
+            ["Top level folder", "sub folder 1", "sub folder 2", ...]
+            returns empty list if album is not in any folders """
+
+        try:
+            return self._folders
+        except AttributeError:
+            self._folders = self._db._album_folder_hierarchy_folderinfo(self._uuid)
+            return self._folders
+
+    @property
+    def parent(self):
+        """ returns FolderInfo object for parent folder or None if no parent (e.g. top-level album) """
+        try:
+            return self._parent
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                parent_uuid = self._db._dbalbum_details[self._uuid]["folderUuid"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=parent_uuid)
+                    if parent_uuid != _PHOTOS_4_TOP_LEVEL_ALBUM
+                    else None
+                )
+            else:
+                parent_pk = self._db._dbalbum_details[self._uuid]["parentfolder"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=self._db._dbalbums_pk[parent_pk])
+                    if parent_pk != self._db._folder_root_pk
+                    else None
+                )
+            return self._parent
+
+    def __len__(self):
+        """ return number of photos contained in album """
+        return len(self.photos)
+
+
+class FolderInfo:
+    """
+    Info about a specific folder, contains all the details about the folder 
+    including folders, albums, etc
+    """
+
+    def __init__(self, db=None, uuid=None):
+        self._uuid = uuid
+        self._db = db
+        if self._db._db_version <= _PHOTOS_4_VERSION:
+            self._pk = None
+            self._title = self._db._dbfolder_details[uuid]["name"]
+        else:
+            self._pk = self._db._dbalbum_details[uuid]["pk"]
+            self._title = self._db._dbalbum_details[uuid]["title"]
+
+    @property
+    def title(self):
+        """ return title / name of folder"""
+        return self._title
+
+    @property
+    def uuid(self):
+        """ return uuid of folder """
+        return self._uuid
+
+    @property
+    def album_info(self):
+        """ return list of albums (as AlbumInfo objects) contained in the folder """
+        try:
+            return self._albums
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                albums = [
+                    AlbumInfo(db=self._db, uuid=album)
+                    for album, detail in self._db._dbalbum_details.items()
+                    if not detail["intrash"]
+                    and detail["albumSubclass"] == _PHOTOS_4_ALBUM_KIND
+                    and detail["folderUuid"] == self._uuid
+                ]
+            else:
+                albums = [
+                    AlbumInfo(db=self._db, uuid=album)
+                    for album, detail in self._db._dbalbum_details.items()
+                    if not detail["intrash"]
+                    and detail["kind"] == _PHOTOS_5_ALBUM_KIND
+                    and detail["parentfolder"] == self._pk
+                ]
+            self._albums = albums
+            return self._albums
+
+    @property
+    def parent(self):
+        """ returns FolderInfo object for parent or None if no parent (e.g. top-level folder) """
+        try:
+            return self._parent
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                parent_uuid = self._db._dbfolder_details[self._uuid]["parentFolderUuid"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=parent_uuid)
+                    if parent_uuid != _PHOTOS_4_TOP_LEVEL_ALBUM
+                    else None
+                )
+            else:
+                parent_pk = self._db._dbalbum_details[self._uuid]["parentfolder"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=self._db._dbalbums_pk[parent_pk])
+                    if parent_pk != self._db._folder_root_pk
+                    else None
+                )
+            return self._parent
+
+    @property
+    def subfolders(self):
+        """ return list of folders (as FolderInfo objects) contained in the folder """
+        try:
+            return self._folders
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                folders = [
+                    FolderInfo(db=self._db, uuid=folder)
+                    for folder, detail in self._db._dbfolder_details.items()
+                    if not detail["intrash"]
+                    and not detail["isMagic"]
+                    and detail["parentFolderUuid"] == self._uuid
+                ]
+            else:
+                folders = [
+                    FolderInfo(db=self._db, uuid=album)
+                    for album, detail in self._db._dbalbum_details.items()
+                    if not detail["intrash"]
+                    and detail["kind"] == _PHOTOS_5_FOLDER_KIND
+                    and detail["parentfolder"] == self._pk
+                ]
+            self._folders = folders
+            return self._folders
+
+    def __len__(self):
+        """ returns count of folders + albums contained in the folder """
+        return len(self.subfolders) + len(self.album_info)

osxphotos/datetime_formatter.py

@@ -0,0 +1,52 @@
+""" Simple formatting of datetime.datetime objects """
+
+import datetime
+
+
+class DateTimeFormatter:
+    """ provides property access to formatted datetime.datetime strftime values """
+
+    def __init__(self, dt: datetime.datetime):
+        self.dt = dt
+
+    @property
+    def date(self):
+        """ ISO date in form 2020-03-22 """
+        date = self.dt.date().isoformat()
+        return date
+
+    @property
+    def year(self):
+        """ 4 digit year """
+        year = f"{self.dt.year}"
+        return year
+
+    @property
+    def yy(self):
+        """ 2 digit year """
+        yy = f"{self.dt.strftime('%y')}"
+        return yy
+
+    @property
+    def mm(self):
+        """ 2 digit month """
+        mm = f"{self.dt.strftime('%m')}"
+        return mm
+
+    @property
+    def month(self):
+        """ Month as locale's full name """
+        month = f"{self.dt.strftime('%B')}"
+        return month
+
+    @property
+    def mon(self):
+        """ Month as locale's abbreviated name """
+        mon = f"{self.dt.strftime('%b')}"
+        return mon
+
+    @property
+    def doy(self):
+        """ Julian day of year starting from 001 """
+        doy = f"{self.dt.strftime('%j')}"
+        return doy

osxphotos/exiftool.py

@@ -0,0 +1,258 @@
+""" Yet another simple exiftool wrapper 
+    I rolled my own for following reasons: 
+    1. I wanted something under MIT license (best alternative was licensed under GPL/BSD)
+    2. I wanted singleton behavior so only a single exiftool process was ever running
+    If these aren't important to you, I highly recommend you use Sven Marnach's excellent 
+    pyexiftool: https://github.com/smarnach/pyexiftool which provides more functionality """
+
+import json
+import logging
+import os
+import subprocess
+import sys
+from functools import lru_cache  # pylint: disable=syntax-error
+
+from .utils import _debug
+
+# exiftool -stay_open commands outputs this EOF marker after command is run
+EXIFTOOL_STAYOPEN_EOF = "{ready}"
+EXIFTOOL_STAYOPEN_EOF_LEN = len(EXIFTOOL_STAYOPEN_EOF)
+
+
+@lru_cache(maxsize=1)
+def get_exiftool_path():
+    """ return path of exiftool, cache result """
+    result = subprocess.run(["which", "exiftool"], stdout=subprocess.PIPE)
+    exiftool_path = result.stdout.decode("utf-8")
+    if _debug():
+        logging.debug("exiftool path = %s" % (exiftool_path))
+    if exiftool_path:
+        return exiftool_path.rstrip()
+    else:
+        raise FileNotFoundError(
+            "Could not find exiftool. Please download and install from "
+            "https://exiftool.org/"
+        )
+
+
+class _ExifToolProc:
+    """ Runs exiftool in a subprocess via Popen
+        Creates a singleton object """
+
+    def __new__(cls, *args, **kwargs):
+        """ create new object or return instance of already created singleton """
+        if not hasattr(cls, "instance") or not cls.instance:
+            cls.instance = super().__new__(cls)
+
+        return cls.instance
+
+    def __init__(self, exiftool=None):
+        """ construct _ExifToolProc singleton object or return instance of already created object
+            exiftool: optional path to exiftool binary (if not provided, will search path to find it) """
+
+        if hasattr(self, "_process_running") and self._process_running:
+            # already running
+            if exiftool is not None:
+                logging.warning(
+                    f"exiftool subprocess already running, "
+                    f"ignoring exiftool={exiftool}"
+                )
+            return
+
+        if exiftool:
+            self._exiftool = exiftool
+        else:
+            self._exiftool = get_exiftool_path()
+
+        self._process_running = False
+        self._start_proc()
+
+    @property
+    def process(self):
+        """ return the exiftool subprocess """
+        if self._process_running:
+            return self._process
+        else:
+            raise ValueError("exiftool process is not running")
+
+    @property
+    def pid(self):
+        """ return process id (PID) of the exiftool process """
+        return self._process.pid
+
+    @property
+    def exiftool(self):
+        """ return path to exiftool process """
+        return self._exiftool
+
+    def _start_proc(self):
+        """ start exiftool in batch mode """
+
+        if self._process_running:
+            logging.warning("exiftool already running: {self._process}")
+            return
+
+        # open exiftool process
+        self._process = subprocess.Popen(
+            [
+                self._exiftool,
+                "-stay_open",  # keep process open in batch mode
+                "True",  # -stay_open=True, keep process open in batch mode
+                "-@",  # read command-line arguments from file
+                "-",  # read from stdin
+                "-common_args",  # specifies args common to all commands subsequently run
+                "-n",  # no print conversion (e.g. print tag values in machine readable format)
+                "-G",  # print group name for each tag
+            ],
+            stdin=subprocess.PIPE,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.DEVNULL,
+        )
+        self._process_running = True
+
+    def _stop_proc(self):
+        """ stop the exiftool process if it's running, otherwise, do nothing """
+        if not self._process_running:
+            logging.warning("exiftool process is not running")
+            return
+
+        self._process.stdin.write(b"-stay_open\n")
+        self._process.stdin.write(b"False\n")
+        self._process.stdin.flush()
+        try:
+            self._process.communicate(timeout=5)
+        except subprocess.TimeoutExpired:
+            logging.warning(
+                f"exiftool pid {self._process.pid} did not exit, killing it"
+            )
+            self._process.kill()
+            self._process.communicate()
+
+        del self._process
+        self._process_running = False
+
+    def __del__(self):
+        self._stop_proc()
+
+
+class ExifTool:
+    """ Basic exiftool interface for reading and writing EXIF tags """
+
+    def __init__(self, filepath, exiftool=None, overwrite=True):
+        """ Return ExifTool object
+            file: path to image file
+            exiftool: path to exiftool, if not specified will look in path
+            overwrite: if True, will overwrite image file without creating backup, default=False """
+        self.file = filepath
+        self.overwrite = overwrite
+        self.data = {}
+        self._exiftoolproc = _ExifToolProc(exiftool=exiftool)
+        self._process = self._exiftoolproc.process
+        self._read_exif()
+
+    def setvalue(self, tag, value):
+        """ Set tag to value(s) 
+            if value is None, will delete tag """
+
+        if value is None:
+            value = ""
+        command = []
+        command.append(f"-{tag}={value}")
+        if self.overwrite:
+            command.append("-overwrite_original")
+        self.run_commands(*command)
+
+    def addvalues(self, tag, *values):
+        """ Add one or more value(s) to tag
+            If more than one value is passed, each value will be added to the tag
+            Notes: exiftool may add duplicate values for some tags so the caller must ensure
+                   the values being added are not already in the EXIF data
+                   For some tags, such as IPTC:Keywords, this will add a new value to the list of keywords, 
+                   but for others, such as EXIF:ISO, this will literally add a value to the existing value.
+                   It's up to the caller to know what exiftool will do for each tag
+                   If setvalue called before addvalues, exiftool does not appear to add duplicates, 
+                   but if addvalues called without first calling setvalue, exiftool will add duplicate values
+        """
+        if not values:
+            raise ValueError("Must pass at least one value")
+
+        command = []
+        for value in values:
+            if value is None:
+                raise ValueError("Can't add None value to tag")
+            command.append(f"-{tag}+={value}")
+
+        if self.overwrite:
+            command.append("-overwrite_original")
+
+        if command:
+            self.run_commands(*command)
+
+    def run_commands(self, *commands, no_file=False):
+        """ run commands in the exiftool process and return result 
+            no_file: (bool) do not pass the filename to exiftool (default=False)
+                     by default, all commands will be run against self.file
+                     use no_file=True to run a command without passing the filename """
+        if not hasattr(self, "_process") or not self._process:
+            raise ValueError("exiftool process is not running")
+
+        if not commands:
+            raise TypeError("must provide one or more command to run")
+
+        filename = os.fsencode(self.file) if not no_file else b""
+        command_str = (
+            b"\n".join([c.encode("utf-8") for c in commands])
+            + b"\n"
+            + filename
+            + b"\n"
+            + b"-execute\n"
+        )
+
+        if _debug():
+            logging.debug(command_str)
+
+        # send the command
+        self._process.stdin.write(command_str)
+        self._process.stdin.flush()
+
+        # read the output
+        output = b""
+        while EXIFTOOL_STAYOPEN_EOF not in str(output):
+            output += self._process.stdout.readline().strip()
+        return output[:-EXIFTOOL_STAYOPEN_EOF_LEN]
+
+    @property
+    def pid(self):
+        """ return process id (PID) of the exiftool process """
+        return self._process.pid
+
+    @property
+    def version(self):
+        """ returns exiftool version """
+        ver = self.run_commands("-ver", no_file=True)
+        return ver.decode("utf-8")
+
+    def as_dict(self):
+        """ return dictionary of all EXIF tags and values from exiftool 
+            returns empty dict if no tags
+        """
+        json_str = self.run_commands("-json")
+        if json_str:
+            exifdict = json.loads(json_str)
+            return exifdict[0]
+        else:
+            return dict()
+
+    def json(self):
+        """ returns JSON string containing all EXIF tags and values from exiftool """
+        json_str = self.run_commands("-json")
+        return json_str
+
+    def _read_exif(self):
+        """ read exif data from file """
+        data = self.as_dict()
+        self.data = {k: v for k, v in data.items()}
+
+    def __str__(self):
+        str_ = f"file: {self.file}\nexiftool: {self._exiftoolproc._exiftool}"
+        return str_

osxphotos/photoinfo.py

@@ -1,604 +0,0 @@
-"""
-PhotoInfo class
-Represents a single photo in the Photos library and provides access to the photo's attributes
-PhotosDB.photos() returns a list of PhotoInfo objects
-"""
-
-import json
-import logging
-import os.path
-import pathlib
-import re
-import subprocess
-import sys
-from datetime import timedelta, timezone
-from pprint import pformat
-
-import yaml
-
-from ._constants import (
-    _MOVIE_TYPE,
-    _PHOTO_TYPE,
-    _PHOTOS_5_SHARED_PHOTO_PATH,
-    _PHOTOS_5_VERSION,
-)
-from .utils import _get_resource_loc, dd_to_dms_str
-
-# TODO: check pylint output
-
-
-class PhotoInfo:
-    """
-    Info about a specific photo, contains all the details about the photo
-    including keywords, persons, albums, uuid, path, etc.
-    """
-
-    def __init__(self, db=None, uuid=None, info=None):
-        self._uuid = uuid
-        self._info = info
-        self._db = db
-
-    @property
-    def filename(self):
-        """ filename of the picture """
-        return self._info["filename"]
-
-    @property
-    def original_filename(self):
-        """ original filename of the picture 
-            Photos 5 mangles filenames upon import """
-        return self._info["originalFilename"]
-
-    @property
-    def date(self):
-        """ image creation date as timezone aware datetime object """
-        imagedate = self._info["imageDate"]
-        seconds = self._info["imageTimeZoneOffsetSeconds"] or 0
-        delta = timedelta(seconds=seconds)
-        tz = timezone(delta)
-        imagedate_utc = imagedate.astimezone(tz=tz)
-        return imagedate_utc
-
-    @property
-    def tzoffset(self):
-        """ timezone offset from UTC in seconds """
-        return self._info["imageTimeZoneOffsetSeconds"]
-
-    @property
-    def path(self):
-        """ absolute path on disk of the original picture """
-
-        photopath = None
-        if self._info["isMissing"] == 1:
-            return photopath  # path would be meaningless until downloaded
-
-        if self._db._db_version < _PHOTOS_5_VERSION:
-            vol = self._info["volume"]
-            if vol is not None:
-                photopath = os.path.join("/Volumes", vol, self._info["imagePath"])
-            else:
-                photopath = os.path.join(
-                    self._db._masters_path, self._info["imagePath"]
-                )
-            return photopath
-            # TODO: Is there a way to use applescript or PhotoKit to force the download in this
-
-        if self._info["shared"]:
-            # shared photo
-            photopath = os.path.join(
-                self._db._library_path,
-                _PHOTOS_5_SHARED_PHOTO_PATH,
-                self._info["directory"],
-                self._info["filename"],
-            )
-            return photopath
-
-        # if self._info["masterFingerprint"]:
-        # if masterFingerprint is not null, path appears to be valid
-        if self._info["directory"].startswith("/"):
-            photopath = os.path.join(self._info["directory"], self._info["filename"])
-        else:
-            photopath = os.path.join(
-                self._db._masters_path, self._info["directory"], self._info["filename"]
-            )
-        return photopath
-
-        # if all else fails, photopath = None
-        # photopath = None
-        # logging.debug(
-        #     f"WARNING: photopath None, masterFingerprint null, not shared {pformat(self._info)}"
-        # )
-        # return photopath
-
-    @property
-    def path_edited(self):
-        """ absolute path on disk of the edited picture """
-        """ None if photo has not been edited """
-        photopath = ""
-
-        if self._db._db_version < _PHOTOS_5_VERSION:
-            if self._info["hasAdjustments"]:
-                edit_id = self._info["edit_resource_id"]
-                if edit_id is not None:
-                    library = self._db._library_path
-                    folder_id, file_id = _get_resource_loc(edit_id)
-                    # todo: is this always true or do we need to search file file_id under folder_id
-                    # figure out what kind it is and build filename
-                    filename = None
-                    if self._info["type"] == _PHOTO_TYPE:
-                        # it's a photo
-                        filename = f"fullsizeoutput_{file_id}.jpeg"
-                    elif self._info["type"] == _MOVIE_TYPE:
-                        # it's a movie
-                        filename = f"fullsizeoutput_{file_id}.mov"
-                    else:
-                        # don't know what it is!
-                        logging.debug(f"WARNING: unknown type {self._info['type']}")
-                        return None
-
-                    photopath = os.path.join(
-                        library,
-                        "resources",
-                        "media",
-                        "version",
-                        folder_id,
-                        "00",
-                        filename,
-                    )
-                    if not os.path.isfile(photopath):
-                        logging.warning(
-                            f"edited file for UUID {self._uuid} should be at {photopath} but does not appear to exist"
-                        )
-                        photopath = None
-                else:
-                    logging.warning(
-                        f"{self.uuid} hasAdjustments but edit_resource_id is None"
-                    )
-            else:
-                photopath = None
-
-            # if self._info["isMissing"] == 1:
-            #     photopath = None  # path would be meaningless until downloaded
-        else:
-            # in Photos 5.0 / Catalina / MacOS 10.15:
-            # edited photos appear to always be converted to .jpeg and stored in
-            # library_name/resources/renders/X/UUID_1_201_a.jpeg
-            # where X = first letter of UUID
-            # and UUID = UUID of image
-            # this seems to be true even for photos not copied to Photos library and
-            # where original format was not jpg/jpeg
-            # if more than one edit, previous edit is stored as UUID_p.jpeg
-
-            if self._info["hasAdjustments"]:
-                library = self._db._library_path
-                directory = self._uuid[0]  # first char of uuid
-                filename = None
-                if self._info["type"] == _PHOTO_TYPE:
-                    # it's a photo
-                    filename = f"{self._uuid}_1_201_a.jpeg"
-                elif self._info["type"] == _MOVIE_TYPE:
-                    # it's a movie
-                    filename = f"{self._uuid}_2_0_a.mov"
-                else:
-                    # don't know what it is!
-                    logging.debug(f"WARNING: unknown type {self._info['type']}")
-                    return None
-
-                photopath = os.path.join(
-                    library, "resources", "renders", directory, filename
-                )
-
-                if not os.path.isfile(photopath):
-                    logging.warning(
-                        f"edited file for UUID {self._uuid} should be at {photopath} but does not appear to exist"
-                    )
-                    photopath = None
-            else:
-                photopath = None
-
-            # TODO: might be possible for original/master to be missing but edit to still be there
-            # if self._info["isMissing"] == 1:
-            #     photopath = None  # path would be meaningless until downloaded
-
-            logging.debug(photopath)
-
-        return photopath
-
-    @property
-    def description(self):
-        """ long / extended description of picture """
-        return self._info["extendedDescription"]
-
-    @property
-    def persons(self):
-        """ list of persons in picture """
-        return self._info["persons"]
-
-    @property
-    def albums(self):
-        """ list of albums picture is contained in """
-        albums = []
-        for album in self._info["albums"]:
-            albums.append(self._db._dbalbum_details[album]["title"])
-        return albums
-
-    @property
-    def keywords(self):
-        """ list of keywords for picture """
-        return self._info["keywords"]
-
-    @property
-    def title(self):
-        """ name / title of picture """
-        # TODO: Update documentation and tests to use title
-        return self._info["name"]
-
-    @property
-    def uuid(self):
-        """ UUID of picture """
-        return self._uuid
-
-    @property
-    def ismissing(self):
-        """ returns true if photo is missing from disk (which means it's not been downloaded from iCloud) 
-        NOTE:   the photos.db database uses an asynchrounous write-ahead log so changes in Photos
-                do not immediately get written to disk. In particular, I've noticed that downloading 
-                an image from the cloud does not force the database to be updated until something else
-                e.g. an edit, keyword, etc. occurs forcing a database synch
-                The exact process / timing is a mystery to be but be aware that if some photos were recently
-                downloaded from cloud to local storate their status in the database might still show
-                isMissing = 1
-        """
-        return True if self._info["isMissing"] == 1 else False
-
-    @property
-    def hasadjustments(self):
-        """ True if picture has adjustments / edits """
-        return True if self._info["hasAdjustments"] == 1 else False
-
-    @property
-    def external_edit(self):
-        """ Returns True if picture was edited outside of Photos using external editor """
-        return (
-            True
-            if self._info["adjustmentFormatID"] == "com.apple.Photos.externalEdit"
-            else False
-        )
-
-    @property
-    def favorite(self):
-        """ True if picture is marked as favorite """
-        return True if self._info["favorite"] == 1 else False
-
-    @property
-    def hidden(self):
-        """ True if picture is hidden """
-        return True if self._info["hidden"] == 1 else False
-
-    @property
-    def location(self):
-        """ returns (latitude, longitude) as float in degrees or None """
-        return (self._latitude, self._longitude)
-
-    @property
-    def shared(self):
-        """ returns True if photos is in a shared iCloud album otherwise false
-            Only valid on Photos 5; returns None on older versions """
-        if self._db._db_version >= _PHOTOS_5_VERSION:
-            return self._info["shared"]
-        else:
-            return None
-
-    @property
-    def uti(self):
-        """ Returns Uniform Type Identifier (UTI) for the image
-            for example: public.jpeg or com.apple.quicktime-movie
-        """
-        return self._info["UTI"]
-
-    @property
-    def ismovie(self):
-        """ Returns True if file is a movie, otherwise False
-        """
-        return True if self._info["type"] == _MOVIE_TYPE else False
-
-    @property
-    def isphoto(self):
-        """ Returns True if file is an image, otherwise False
-        """
-        return True if self._info["type"] == _PHOTO_TYPE else False
-
-    @property
-    def burst(self):
-        """ Returns True if photo is part of a Burst photo set, otherwise False """
-        return self._info["burst"]
-
-    @property
-    def burst_photos(self):
-        """ If photo is a burst photo, returns list of PhotoInfo objects 
-            that are part of the same burst photo set; otherwise returns empty list.
-            self is not included in the returned list """
-        if self._info["burst"]:
-            burst_uuid = self._info["burstUUID"]
-            burst_photos = [
-                PhotoInfo(db=self._db, uuid=u, info=self._db._dbphotos[u])
-                for u in self._db._dbphotos_burst[burst_uuid]
-                if u != self._uuid
-            ]
-            return burst_photos
-        else:
-            return []
-
-    def export(
-        self,
-        dest,
-        *filename,
-        edited=False,
-        overwrite=False,
-        increment=True,
-        sidecar=False,
-    ):
-        """ export photo """
-        """ first argument must be valid destination path (or exception raised) """
-        """ second argument (optional): name of picture; if not provided, will use current filename """
-        """ if edited=True (default=False), will export the edited version of the photo (or raise exception if no edited version) """
-        """ if overwrite=True (default=False), will overwrite files if they alreay exist """
-        """ if increment=True (default=True), will increment file name until a non-existant name is found """
-        """ if overwrite=False and increment=False, export will fail if destination file already exists """
-        """ if sidecar=True, will also write a json sidecar with EXIF data in format readable by exiftool """
-        """ sidecar filename will be dest/filename.ext.json where ext is suffix of the image file (e.g. jpeg or jpg) """
-        """ returns the full path to the exported file """
-
-        # TODO: add this docs:
-        #  ( for jpeg in *.jpeg; do exiftool -v -json=$jpeg.json $jpeg; done )
-
-        # check arguments and get destination path and filename (if provided)
-        if filename and len(filename) > 2:
-            raise TypeError(
-                "Too many positional arguments.  Should be at most two: destination, filename."
-            )
-        else:
-            # verify destination is a valid path
-            if dest is None:
-                raise ValueError("Destination must not be None")
-            elif not os.path.isdir(dest):
-                raise FileNotFoundError("Invalid path passed to export")
-
-            if filename and len(filename) == 1:
-                # second arg is filename of picture
-                filename = filename[0]
-            else:
-                # no filename provided so use the default
-                # if edited file requested, use filename but add _edited
-                # need to use file extension from edited file as Photos saves a jpeg once edited
-                if edited:
-                    # verify we have a valid path_edited and use that to get filename
-                    if not self.path_edited:
-                        raise FileNotFoundError(
-                            "edited=True but path_edited is none; hasadjustments: "
-                            f" {self.hasadjustments}"
-                        )
-                    edited_name = pathlib.Path(self.path_edited).name
-                    edited_suffix = pathlib.Path(edited_name).suffix
-                    filename = (
-                        pathlib.Path(self.filename).stem + "_edited" + edited_suffix
-                    )
-                else:
-                    filename = self.filename
-
-        # get path to source file and verify it's not None and is valid file
-        # TODO: how to handle ismissing or not hasadjustments and edited=True cases?
-        if edited:
-            if not self.hasadjustments:
-                logging.warning(
-                    "Attempting to export edited photo but hasadjustments=False"
-                )
-
-            if self.path_edited is not None:
-                src = self.path_edited
-            else:
-                raise FileNotFoundError(
-                    f"edited=True but path_edited is none; hasadjustments: {self.hasadjustments}"
-                )
-        else:
-            if self.ismissing:
-                logging.warning(
-                    f"Attempting to export photo with ismissing=True: path = {self.path}"
-                )
-
-            if self.path is None:
-                logging.warning(
-                    f"Attempting to export photo but path is None: ismissing = {self.ismissing}"
-                )
-                raise FileNotFoundError("Cannot export photo if path is None")
-            else:
-                src = self.path
-
-        if not os.path.isfile(src):
-            raise FileNotFoundError(f"{src} does not appear to exist")
-
-        dest = pathlib.Path(dest)
-        filename = pathlib.Path(filename)
-        dest = dest / filename
-
-        # check to see if file exists and if so, add (1), (2), etc until we find one that works
-        if increment and not overwrite:
-            count = 1
-            dest_new = dest
-            while dest_new.exists():
-                dest_new = dest.parent / f"{dest.stem} ({count}){dest.suffix}"
-                count += 1
-            dest = dest_new
-
-        logging.debug(
-            f"exporting {src} to {dest}, overwrite={overwrite}, incremetn={increment}, dest exists: {dest.exists()}"
-        )
-
-        # if overwrite==False and #increment==False, export should fail if file exists
-        if dest.exists() and not overwrite and not increment:
-            raise FileExistsError(
-                f"destination exists ({dest}); overwrite={overwrite}, increment={increment}"
-            )
-
-        # if error on copy, subprocess will raise CalledProcessError
-        try:
-            subprocess.run(
-                ["/usr/bin/ditto", src, dest], check=True, stderr=subprocess.PIPE
-            )
-        except subprocess.CalledProcessError as e:
-            logging.critical(
-                f"ditto returned error: {e.returncode} {e.stderr.decode(sys.getfilesystemencoding()).rstrip()}"
-            )
-            raise e
-
-        if sidecar:
-            logging.debug("writing exiftool_json_sidecar")
-            sidecar_filename = f"{dest}.json"
-            json_sidecar_str = self._exiftool_json_sidecar()
-            try:
-                self._write_sidecar_car(sidecar_filename, json_sidecar_str)
-            except Exception as e:
-                logging.critical(f"Error writing json sidecar to {sidecar_filename}")
-                raise e
-
-        return str(dest)
-
-    def _exiftool_json_sidecar(self):
-        """ return json string of EXIF details in exiftool sidecar format """
-        exif = {}
-        exif["FileName"] = self.filename
-
-        if self.description:
-            exif["ImageDescription"] = self.description
-            exif["Description"] = self.description
-
-        if self.title:
-            exif["Title"] = self.title
-
-        if self.keywords:
-            exif["TagsList"] = exif["Keywords"] = self.keywords
-
-        if self.persons:
-            exif["PersonInImage"] = self.persons
-
-        # if self.favorite():
-        #     exif["Rating"] = 5
-
-        (lat, lon) = self.location
-        if lat is not None and lon is not None:
-            lat_str, lon_str = dd_to_dms_str(lat, lon)
-            exif["GPSLatitude"] = lat_str
-            exif["GPSLongitude"] = lon_str
-            exif["GPSPosition"] = f"{lat_str}, {lon_str}"
-            lat_ref = "North" if lat >= 0 else "South"
-            lon_ref = "East" if lon >= 0 else "West"
-            exif["GPSLatitudeRef"] = lat_ref
-            exif["GPSLongitudeRef"] = lon_ref
-
-        # process date/time and timezone offset
-        date = self.date
-        # exiftool expects format to "2015:01:18 12:00:00"
-        datetimeoriginal = date.strftime("%Y:%m:%d %H:%M:%S")
-        offsettime = date.strftime("%z")
-        # find timezone offset in format "-04:00"
-        offset = re.findall(r"([+-]?)([\d]{2})([\d]{2})", offsettime)
-        offset = offset[0]  # findall returns list of tuples
-        offsettime = f"{offset[0]}{offset[1]}:{offset[2]}"
-        exif["DateTimeOriginal"] = datetimeoriginal
-        exif["OffsetTimeOriginal"] = offsettime
-
-        json_str = json.dumps([exif])
-        return json_str
-
-    def _write_sidecar_car(self, filename, json_str):
-        if not filename and not json_str:
-            raise (
-                ValueError(
-                    f"filename {filename} and json_str {json_str} must not be None"
-                )
-            )
-
-        # TODO: catch exception?
-        f = open(filename, "w")
-        f.write(json_str)
-        f.close()
-
-    @property
-    def _longitude(self):
-        """ Returns longitude, in degrees """
-        return self._info["longitude"]
-
-    @property
-    def _latitude(self):
-        """ Returns latitude, in degrees """
-        return self._info["latitude"]
-
-    def __repr__(self):
-        return f"osxphotos.{self.__class__.__name__}(db={self._db}, uuid='{self._uuid}', info={self._info})"
-
-    def __str__(self):
-        info = {
-            "uuid": self.uuid,
-            "filename": self.filename,
-            "original_filename": self.original_filename,
-            "date": str(self.date),
-            "description": self.description,
-            "title": self.title,
-            "keywords": self.keywords,
-            "albums": self.albums,
-            "persons": self.persons,
-            "path": self.path,
-            "ismissing": self.ismissing,
-            "hasadjustments": self.hasadjustments,
-            "external_edit": self.external_edit,
-            "favorite": self.favorite,
-            "hidden": self.hidden,
-            "latitude": self._latitude,
-            "longitude": self._longitude,
-            "path_edited": self.path_edited,
-            "shared": self.shared,
-            "isphoto": self.isphoto,
-            "ismovie": self.ismovie,
-            "uti": self.uti,
-            "burst": self.burst,
-        }
-        return yaml.dump(info, sort_keys=False)
-
-    def json(self):
-        """ return JSON representation """
-        pic = {
-            "uuid": self.uuid,
-            "filename": self.filename,
-            "original_filename": self.original_filename,
-            "date": self.date.isoformat(),
-            "description": self.description,
-            "title": self.title,
-            "keywords": self.keywords,
-            "albums": self.albums,
-            "persons": self.persons,
-            "path": self.path,
-            "ismissing": self.ismissing,
-            "hasadjustments": self.hasadjustments,
-            "external_edit": self.external_edit,
-            "favorite": self.favorite,
-            "hidden": self.hidden,
-            "latitude": self._latitude,
-            "longitude": self._longitude,
-            "path_edited": self.path_edited,
-            "shared": self.shared,
-            "isphoto": self.isphoto,
-            "ismovie": self.ismovie,
-            "uti": self.uti,
-            "burst": self.burst,
-        }
-        return json.dumps(pic)
-
-    # compare two PhotoInfo objects for equality
-    def __eq__(self, other):
-        if isinstance(other, self.__class__):
-            return self.__dict__ == other.__dict__
-
-        return False
-
-    def __ne__(self, other):
-        return not self.__eq__(other)

osxphotos/photoinfo/__init__.py

@@ -0,0 +1,8 @@
+"""
+PhotoInfo class
+Represents a single photo in the Photos library and provides access to the photo's attributes
+PhotosDB.photos() returns a list of PhotoInfo objects
+"""
+
+from .photoinfo import PhotoInfo
+from ._photoinfo_exifinfo import ExifInfo

osxphotos/photoinfo/_photoinfo_exifinfo.py

@@ -0,0 +1,94 @@
+""" PhotoInfo methods to expose EXIF info from the library """
+
+import logging
+from dataclasses import dataclass
+
+from .._constants import _PHOTOS_4_VERSION
+
+
+@dataclass(frozen=True)
+class ExifInfo:
+    """ EXIF info associated with a photo from the Photos library """
+
+    flash_fired: bool
+    iso: int
+    metering_mode: int
+    sample_rate: int
+    track_format: int
+    white_balance: int
+    aperture: float
+    bit_rate: float
+    duration: float
+    exposure_bias: float
+    focal_length: float
+    fps: float
+    latitude: float
+    longitude: float
+    shutter_speed: float
+    camera_make: str
+    camera_model: str
+    codec: str
+    lens_model: str
+
+
+@property
+def exif_info(self):
+    """ Returns an ExifInfo object with the EXIF data for photo
+        Note: the returned EXIF data is the data Photos stores in the database on import;
+        ExifInfo does not provide access to the EXIF info in the actual image file
+        Some or all of the fields may be None
+        Only valid for Photos 5; on earlier database returns None
+        """
+
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        logging.debug(f"exif_info not implemented for this database version")
+        return None
+
+    try:
+        exif = self._db._db_exifinfo_uuid[self.uuid]
+        exif_info = ExifInfo(
+            iso=exif["ZISO"],
+            flash_fired=True if exif["ZFLASHFIRED"] == 1 else False,
+            metering_mode=exif["ZMETERINGMODE"],
+            sample_rate=exif["ZSAMPLERATE"],
+            track_format=exif["ZTRACKFORMAT"],
+            white_balance=exif["ZWHITEBALANCE"],
+            aperture=exif["ZAPERTURE"],
+            bit_rate=exif["ZBITRATE"],
+            duration=exif["ZDURATION"],
+            exposure_bias=exif["ZEXPOSUREBIAS"],
+            focal_length=exif["ZFOCALLENGTH"],
+            fps=exif["ZFPS"],
+            latitude=exif["ZLATITUDE"],
+            longitude=exif["ZLONGITUDE"],
+            shutter_speed=exif["ZSHUTTERSPEED"],
+            camera_make=exif["ZCAMERAMAKE"],
+            camera_model=exif["ZCAMERAMODEL"],
+            codec=exif["ZCODEC"],
+            lens_model=exif["ZLENSMODEL"],
+        )
+    except KeyError:
+        logging.debug(f"Could not find exif record for uuid {self.uuid}")
+        exif_info = ExifInfo(
+            iso=None,
+            flash_fired=None,
+            metering_mode=None,
+            sample_rate=None,
+            track_format=None,
+            white_balance=None,
+            aperture=None,
+            bit_rate=None,
+            duration=None,
+            exposure_bias=None,
+            focal_length=None,
+            fps=None,
+            latitude=None,
+            longitude=None,
+            shutter_speed=None,
+            camera_make=None,
+            camera_model=None,
+            codec=None,
+            lens_model=None,
+        )
+
+    return exif_info

osxphotos/photoinfo/_photoinfo_exiftool.py

@@ -0,0 +1,33 @@
+""" Implementation for PhotoInfo.exiftool property which returns ExifTool object for a photo """
+
+import logging
+import os
+
+from ..exiftool import ExifTool, get_exiftool_path
+
+@property
+def exiftool(self):
+    """ Returns an ExifTool object for the photo
+        requires that exiftool (https://exiftool.org/) be installed
+        If exiftool not installed, logs warning and returns None
+        If photo path is missing, returns None
+    """
+    try:
+        # return the memoized instance if it exists
+        return self._exiftool
+    except AttributeError:
+        try:
+            exiftool_path = get_exiftool_path()
+            if self.path is not None and os.path.isfile(self.path):
+                exiftool = ExifTool(self.path)
+            else:
+                exiftool = None
+                logging.debug(f"exiftool: missing path {self.uuid}")
+        except FileNotFoundError:
+            # get_exiftool_path raises FileNotFoundError if exiftool not found
+            exiftool = None
+            logging.warning(f"exiftool not in path; download and install from https://exiftool.org/")
+
+        self._exiftool = exiftool
+        return self._exiftool
+

osxphotos/photoinfo/_photoinfo_searchinfo.py

@@ -0,0 +1,93 @@
+""" Methods and class for PhotoInfo exposing SearchInfo data such as labels 
+    Adds the following properties to PhotoInfo (valid only for Photos 5):
+        search_info: returns a SearchInfo object
+        labels: returns list of labels
+        labels_normalized: returns list of normalized labels
+"""
+
+from .._constants import _PHOTOS_4_VERSION, SEARCH_CATEGORY_LABEL
+
+
+@property
+def search_info(self):
+    """ returns SearchInfo object for photo 
+        only valid on Photos 5, on older libraries, returns None
+    """
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        return None
+
+    # memoize SearchInfo object
+    try:
+        return self._search_info
+    except AttributeError:
+        self._search_info = SearchInfo(self)
+        return self._search_info
+
+
+@property
+def labels(self):
+    """ returns list of labels applied to photo by Photos image categorization
+        only valid on Photos 5, on older libraries returns empty list 
+    """
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        return []
+
+    return self.search_info.labels
+
+
+@property
+def labels_normalized(self):
+    """ returns normalized list of labels applied to photo by Photos image categorization
+        only valid on Photos 5, on older libraries returns empty list 
+    """
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        return []
+
+    return self.search_info.labels_normalized
+
+
+class SearchInfo:
+    """ Info about search terms such as machine learning labels that Photos knows about a photo """
+
+    def __init__(self, photo):
+        """ photo: PhotoInfo object """
+
+        if photo._db._db_version <= _PHOTOS_4_VERSION:
+            raise NotImplementedError(
+                f"search info not implemented for this database version"
+            )
+
+        self._photo = photo
+        self.uuid = photo.uuid
+        try:
+            # get search info for this UUID
+            # there might not be any search info data (e.g. if Photo was missing or photoanalysisd not run yet)
+            self._db_searchinfo = photo._db._db_searchinfo_uuid[self.uuid]
+        except KeyError:
+            self._db_searchinfo = None
+
+    @property
+    def labels(self):
+        """ return list of labels associated with Photo """
+        if self._db_searchinfo:
+            labels = [
+                rec["content_string"]
+                for rec in self._db_searchinfo
+                if rec["category"] == SEARCH_CATEGORY_LABEL
+            ]
+        else:
+            labels = []
+        return labels
+
+    @property
+    def labels_normalized(self):
+        """ return list of normalized labels associated with Photo """
+        if self._db_searchinfo:
+            labels = [
+                rec["normalized_string"]
+                for rec in self._db_searchinfo
+                if rec["category"] == SEARCH_CATEGORY_LABEL
+            ]
+        else:
+            labels = []
+        return labels

osxphotos/photosdb/__init__.py

@@ -0,0 +1,6 @@
+"""
+PhotosDB class
+Processes a Photos.app library database to extract information about photos
+"""
+
+from .photosdb import PhotosDB

osxphotos/photosdb/_photosdb_process_exif.py

@@ -0,0 +1,56 @@
+""" PhotosDB method for processing exif info 
+    Do not import this module directly """
+
+import logging
+
+from .._constants import _PHOTOS_4_VERSION
+from ..utils import _db_is_locked, _debug, _open_sql_file
+
+
+def _process_exifinfo(self):
+    """ load the exif data from the database 
+        this is a PhotosDB method that should be imported in 
+        the PhotosDB class definition in photosdb.py
+    """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        _process_exifinfo_4(self)
+    else:
+        _process_exifinfo_5(self)
+
+
+# The following methods do not get imported into PhotosDB
+# but will get called by _process_exifinfo
+
+
+def _process_exifinfo_4(photosdb):
+    """ process exif info for Photos <= 4 
+        photosdb: PhotosDB instance """
+    photosdb._db_exifinfo_uuid = {}
+    raise NotImplementedError(f"search info not implemented for this database version")
+
+
+def _process_exifinfo_5(photosdb):
+    """ process exif info for Photos >= 5 
+        photosdb: PhotosDB instance """
+
+    db = photosdb._tmp_db
+    
+    (conn, cursor) = _open_sql_file(db)
+
+    result = conn.execute(
+        """
+        SELECT ZGENERICASSET.ZUUID, ZEXTENDEDATTRIBUTES.*
+        FROM ZGENERICASSET
+        JOIN ZEXTENDEDATTRIBUTES
+        ON ZEXTENDEDATTRIBUTES.ZASSET = ZGENERICASSET.Z_PK
+        """
+    )
+
+    photosdb._db_exifinfo_uuid = {}
+    cols = [c[0] for c in result.description]
+    for row in result.fetchall():
+        record = dict(zip(cols, row))
+        uuid = record["ZUUID"]
+        if uuid in photosdb._db_exifinfo_uuid:
+            logging.warning(f"duplicate exifinfo record found for uuid {uuid}")
+        photosdb._db_exifinfo_uuid[uuid] = record

osxphotos/photosdb/_photosdb_process_searchinfo.py

@@ -0,0 +1,197 @@
+""" Methods for PhotosDB to add Photos 5 search info such as machine learning labels 
+    Kudos to Simon Willison who figured out how to extract this data from psi.sql
+    ref: https://github.com/dogsheep/photos-to-sqlite/issues/16
+"""
+
+import logging
+import pathlib
+import uuid as uuidlib
+from pprint import pformat
+
+from .._constants import _PHOTOS_4_VERSION, SEARCH_CATEGORY_LABEL
+from ..utils import _db_is_locked, _debug, _open_sql_file
+
+"""
+    This module should be imported in the class defintion of PhotosDB in photosdb.py
+    Do not import this module directly
+    This module adds the following method to PhotosDB:
+        _process_searchinfo: process search terms from psi.sqlite 
+
+    The following properties are added to PhotosDB    
+        labels: list of all labels in the library
+        labels_normalized: list of all labels normalized in the library  
+        labels_as_dict: dict of {label: count of photos} in reverse sorted order (most photos first)
+        labels_normalized_as_dict: dict of {normalized label: count of photos} in reverse sorted order (most photos first)
+
+    The following data structures are added to PhotosDB
+        self._db_searchinfo_categories
+        self._db_searchinfo_uuid
+        self._db_searchinfo_labels 
+        self._db_searchinfo_labels_normalized 
+    
+    These methods only work on Photos 5 databases.  Will print warning on earlier library versions.
+"""
+
+
+def _process_searchinfo(self):
+    """ load machine learning/search term label info from a Photos library
+        db_connection: a connection to the SQLite database file containing the 
+        search terms.  In Photos 5, this is called psi.sqlite
+        Note: Only works on Photos version == 5.0 """
+
+    if self._db_version <= _PHOTOS_4_VERSION:
+        raise NotImplementedError(
+            f"search info not implemented for this database version"
+        )
+
+    search_db_path = pathlib.Path(self._dbfile).parent / "search" / "psi.sqlite"
+    if not search_db_path.exists():
+        raise FileNotFoundError(f"could not find search db: {search_db_path}")
+
+    if _db_is_locked(search_db_path):
+        search_db = self._copy_db_file(search_db_path)
+    else:
+        search_db = search_db_path
+
+    (conn, c) = _open_sql_file(search_db)
+
+    result = conn.execute(
+        """
+        select
+        ga.rowid,
+        assets.uuid_0,
+        assets.uuid_1,
+        groups.rowid as groupid,
+        groups.category,
+        groups.owning_groupid,
+        groups.content_string,
+        groups.normalized_string,
+        groups.lookup_identifier
+        from
+        ga
+        join groups on groups.rowid = ga.groupid
+        join assets on ga.assetid = assets.rowid
+        order by
+        ga.rowid
+        """
+    )
+
+    # _db_searchinfo_uuid is dict in form {uuid : [list of associated search info records]
+    _db_searchinfo_uuid = {}
+
+    # _db_searchinfo_categories is dict in form {search info category id: list normalized strings for the category
+    # right now, this is mostly for debugging to easily see which search terms are in the library
+    _db_searchinfo_categories = {}
+
+    # _db_searchinfo_labels is dict in form {normalized label: [list of photo uuids]}
+    # this serves as a reverse index from label to photos containing the label
+    # _db_searchinfo_labels_normalized is the same but with normalized (lower case) version of the label
+    _db_searchinfo_labels = {}
+    _db_searchinfo_labels_normalized = {}
+
+    cols = [c[0] for c in result.description]
+    for row in result.fetchall():
+        record = dict(zip(cols, row))
+        uuid = ints_to_uuid(record["uuid_0"], record["uuid_1"])
+        # strings have null character appended, so strip it
+        for key in record:
+            if isinstance(record[key], str):
+                record[key] = record[key].replace("\x00", "")
+        try:
+            _db_searchinfo_uuid[uuid].append(record)
+        except KeyError:
+            _db_searchinfo_uuid[uuid] = [record]
+
+        category = record["category"]
+        try:
+            _db_searchinfo_categories[record["category"]].append(
+                record["normalized_string"]
+            )
+        except KeyError:
+            _db_searchinfo_categories[record["category"]] = [
+                record["normalized_string"]
+            ]
+
+        if record["category"] == SEARCH_CATEGORY_LABEL:
+            label = record["content_string"]
+            label_norm = record["normalized_string"]
+            try:
+                _db_searchinfo_labels[label].append(uuid)
+                _db_searchinfo_labels_normalized[label_norm].append(uuid)
+            except KeyError:
+                _db_searchinfo_labels[label] = [uuid]
+                _db_searchinfo_labels_normalized[label_norm] = [uuid]
+
+    self._db_searchinfo_categories = _db_searchinfo_categories
+    self._db_searchinfo_uuid = _db_searchinfo_uuid
+    self._db_searchinfo_labels = _db_searchinfo_labels
+    self._db_searchinfo_labels_normalized = _db_searchinfo_labels_normalized
+
+    if _debug():
+        logging.debug(
+            "_db_searchinfo_categories: \n" + pformat(self._db_searchinfo_categories)
+        )
+        logging.debug("_db_searchinfo_uuid: \n" + pformat(self._db_searchinfo_uuid))
+        logging.debug("_db_searchinfo_labels: \n" + pformat(self._db_searchinfo_labels))
+        logging.debug(
+            "_db_searchinfo_labels_normalized: \n"
+            + pformat(self._db_searchinfo_labels_normalized)
+        )
+
+
+@property
+def labels(self):
+    """ return list of all search info labels found in the library """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return []
+
+    return list(self._db_searchinfo_labels.keys())
+
+
+@property
+def labels_normalized(self):
+    """ return list of all normalized search info labels found in the library """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return []
+
+    return list(self._db_searchinfo_labels_normalized.keys())
+
+
+@property
+def labels_as_dict(self):
+    """ return labels as dict of label: count in reverse sorted order (descending) """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return dict()
+
+    labels = {k: len(v) for k, v in self._db_searchinfo_labels.items()}
+    labels = dict(sorted(labels.items(), key=lambda kv: kv[1], reverse=True))
+    return labels
+
+
+@property
+def labels_normalized_as_dict(self):
+    """ return normalized labels as dict of label: count in reverse sorted order (descending) """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return dict()
+    labels = {k: len(v) for k, v in self._db_searchinfo_labels_normalized.items()}
+    labels = dict(sorted(labels.items(), key=lambda kv: kv[1], reverse=True))
+    return labels
+
+
+# The following method is not imported into PhotosDB
+
+
+def ints_to_uuid(uuid_0, uuid_1):
+    """ convert two signed ints into a UUID strings
+        uuid_0, uuid_1: the two int components of an RFC 4122 UUID """
+
+    # assumes uuid imported as uuidlib (to avoid namespace conflict with other uses of uuid)
+
+    bytes_ = uuid_0.to_bytes(8, "little", signed=True) + uuid_1.to_bytes(
+        8, "little", signed=True
+    )
+    return str(uuidlib.UUID(bytes=bytes_)).upper()

osxphotos/placeinfo.py

@@ -0,0 +1,626 @@
+""" 
+    PlaceInfo class
+    Provides reverse geolocation info for photos 
+    
+    See https://developer.apple.com/documentation/corelocation/clplacemark
+    for additional documentation on reverse geolocation data
+"""
+from abc import ABC, abstractmethod
+from collections import namedtuple  # pylint: disable=syntax-error
+
+import yaml
+from bpylist import archiver
+
+# postal address information, returned by PlaceInfo.address
+PostalAddress = namedtuple(
+    "PostalAddress",
+    [
+        "street",
+        "sub_locality",
+        "city",
+        "sub_administrative_area",
+        "state_province",
+        "postal_code",
+        "country",
+        "iso_country_code",
+    ],
+)
+
+# PlaceNames tuple returned by PlaceInfo.names
+# order of fields 0 - 17 is mapped to placeType value in
+# PLRevGeoLocationInfo.mapInfo.sortedPlaceInfos
+# field 18 is combined bodies of water (ocean + inland_water)
+# and maps to Photos <= 4, RKPlace.type == 44
+# (Photos <= 4 doesn't have ocean or inland_water types)
+# The fields named "field0", etc. appear to be unused
+PlaceNames = namedtuple(
+    "PlaceNames",
+    [
+        "field0",
+        "country",  # The name of the country associated with the placemark.
+        "state_province",  # administrativeArea, The state or province associated with the placemark.
+        "sub_administrative_area",  # Additional administrative area information for the placemark.
+        "city",  # locality, The city associated with the placemark.
+        "field5",
+        "additional_city_info",  # subLocality, Additional city-level information for the placemark.
+        "ocean",  # The name of the ocean associated with the placemark.
+        "area_of_interest",  # areasOfInterest, The relevant areas of interest associated with the placemark.
+        "inland_water",  # The name of the inland water body associated with the placemark.
+        "field10",
+        "region",  # The geographic region associated with the placemark.
+        "sub_throughfare",  # Additional street-level information for the placemark.
+        "field13",
+        "postal_code",  # The postal code associated with the placemark.
+        "field15",
+        "field16",
+        "street_address",  # throughfare, The street address associated with the placemark.
+        "body_of_water",  # RKPlace.type == 44, appears to be any body of water (ocean or inland)
+    ],
+)
+
+# The following classes represent Photo Library Reverse Geolocation Info as stored
+# in ZADDITIONALASSETATTRIBUTES.ZREVERSELOCATIONDATA
+# These classes are used by bpylist.archiver to unarchive the serialized objects
+class PLRevGeoLocationInfo:
+    """ The top level reverse geolocation object """
+
+    def __init__(
+        self,
+        addressString,
+        countryCode,
+        mapItem,
+        isHome,
+        compoundNames,
+        compoundSecondaryNames,
+        version,
+        geoServiceProvider,
+        postalAddress,
+    ):
+        self.addressString = addressString
+        self.countryCode = countryCode
+        self.mapItem = mapItem
+        self.isHome = isHome
+        self.compoundNames = compoundNames
+        self.compoundSecondaryNames = compoundSecondaryNames
+        self.version = version
+        self.geoServiceProvider = geoServiceProvider
+        self.postalAddress = postalAddress
+
+    def __eq__(self, other):
+        for field in [
+            "addressString",
+            "countryCode",
+            "isHome",
+            "compoundNames",
+            "compoundSecondaryNames",
+            "version",
+            "geoServiceProvider",
+            "postalAddress",
+        ]:
+            if getattr(self, field) != getattr(other, field):
+                return False
+        return True
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __str__(self):
+        return f"addressString: {self.addressString}, countryCode: {self.countryCode}, isHome: {self.isHome}, mapItem: {self.mapItem}, postalAddress: {self.postalAddress}"
+
+    @staticmethod
+    def encode_archive(obj, archive):
+        archive.encode("addressString", obj.addressString)
+        archive.encode("countryCode", obj.countryCode)
+        archive.encode("mapItem", obj.mapItem)
+        archive.encode("isHome", obj.isHome)
+        archive.encode("compoundNames", obj.compoundNames)
+        archive.encode("compoundSecondaryNames", obj.compoundSecondaryNames)
+        archive.encode("version", obj.version)
+        archive.encode("geoServiceProvider", obj.geoServiceProvider)
+        archive.encode("postalAddress", obj.postalAddress)
+
+    @staticmethod
+    def decode_archive(archive):
+        addressString = archive.decode("addressString")
+        countryCode = archive.decode("countryCode")
+        mapItem = archive.decode("mapItem")
+        isHome = archive.decode("isHome")
+        compoundNames = archive.decode("compoundNames")
+        compoundSecondaryNames = archive.decode("compoundSecondaryNames")
+        version = archive.decode("version")
+        geoServiceProvider = archive.decode("geoServiceProvider")
+        postalAddress = archive.decode("postalAddress")
+        return PLRevGeoLocationInfo(
+            addressString,
+            countryCode,
+            mapItem,
+            isHome,
+            compoundNames,
+            compoundSecondaryNames,
+            version,
+            geoServiceProvider,
+            postalAddress,
+        )
+
+
+class PLRevGeoMapItem:
+    """ Stores the list of place names, organized by area """
+
+    def __init__(self, sortedPlaceInfos, finalPlaceInfos):
+        self.sortedPlaceInfos = sortedPlaceInfos
+        self.finalPlaceInfos = finalPlaceInfos
+
+    def __eq__(self, other):
+        for field in ["sortedPlaceInfos", "finalPlaceInfos"]:
+            if getattr(self, field) != getattr(other, field):
+                return False
+        return True
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __str__(self):
+        sortedPlaceInfos = []
+        finalPlaceInfos = []
+        for place in self.sortedPlaceInfos:
+            sortedPlaceInfos.append(str(place))
+        for place in self.finalPlaceInfos:
+            finalPlaceInfos.append(str(place))
+        return (
+            f"finalPlaceInfos: {finalPlaceInfos}, sortedPlaceInfos: {sortedPlaceInfos}"
+        )
+
+    @staticmethod
+    def encode_archive(obj, archive):
+        archive.encode("sortedPlaceInfos", obj.sortedPlaceInfos)
+        archive.encode("finalPlaceInfos", obj.finalPlaceInfos)
+
+    @staticmethod
+    def decode_archive(archive):
+        sortedPlaceInfos = archive.decode("sortedPlaceInfos")
+        finalPlaceInfos = archive.decode("finalPlaceInfos")
+        return PLRevGeoMapItem(sortedPlaceInfos, finalPlaceInfos)
+
+
+class PLRevGeoMapItemAdditionalPlaceInfo:
+    """ Additional info about individual places """
+
+    def __init__(self, area, name, placeType, dominantOrderType):
+        self.area = area
+        self.name = name
+        self.placeType = placeType
+        self.dominantOrderType = dominantOrderType
+
+    def __eq__(self, other):
+        for field in ["area", "name", "placeType", "dominantOrderType"]:
+            if getattr(self, field) != getattr(other, field):
+                return False
+        return True
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __str__(self):
+        return f"area: {self.area}, name: {self.name}, placeType: {self.placeType}"
+
+    @staticmethod
+    def encode_archive(obj, archive):
+        archive.encode("area", obj.area)
+        archive.encode("name", obj.name)
+        archive.encode("placeType", obj.placeType)
+        archive.encode("dominantOrderType", obj.dominantOrderType)
+
+    @staticmethod
+    def decode_archive(archive):
+        area = archive.decode("area")
+        name = archive.decode("name")
+        placeType = archive.decode("placeType")
+        dominantOrderType = archive.decode("dominantOrderType")
+        return PLRevGeoMapItemAdditionalPlaceInfo(
+            area, name, placeType, dominantOrderType
+        )
+
+
+class CNPostalAddress:
+    """ postal address for the reverse geolocation info """
+
+    def __init__(
+        self,
+        _ISOCountryCode,
+        _city,
+        _country,
+        _postalCode,
+        _state,
+        _street,
+        _subAdministrativeArea,
+        _subLocality,
+    ):
+        self._ISOCountryCode = _ISOCountryCode
+        self._city = _city
+        self._country = _country
+        self._postalCode = _postalCode
+        self._state = _state
+        self._street = _street
+        self._subAdministrativeArea = _subAdministrativeArea
+        self._subLocality = _subLocality
+
+    def __eq__(self, other):
+        for field in [
+            "_ISOCountryCode",
+            "_city",
+            "_country",
+            "_postalCode",
+            "_state",
+            "_street",
+            "_subAdministrativeArea",
+            "_subLocality",
+        ]:
+            if getattr(self, field) != getattr(other, field):
+                return False
+        return True
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __str__(self):
+        return ", ".join(
+            map(
+                str,
+                [
+                    self._street,
+                    self._city,
+                    self._subLocality,
+                    self._subAdministrativeArea,
+                    self._state,
+                    self._postalCode,
+                    self._country,
+                    self._ISOCountryCode,
+                ],
+            )
+        )
+
+    @staticmethod
+    def encode_archive(obj, archive):
+        archive.encode("_ISOCountryCode", obj._ISOCountryCode)
+        archive.encode("_country", obj._country)
+        archive.encode("_city", obj._city)
+        archive.encode("_postalCode", obj._postalCode)
+        archive.encode("_state", obj._state)
+        archive.encode("_street", obj._street)
+        archive.encode("_subAdministrativeArea", obj._subAdministrativeArea)
+        archive.encode("_subLocality", obj._subLocality)
+
+    @staticmethod
+    def decode_archive(archive):
+        _ISOCountryCode = archive.decode("_ISOCountryCode")
+        _country = archive.decode("_country")
+        _city = archive.decode("_city")
+        _postalCode = archive.decode("_postalCode")
+        _state = archive.decode("_state")
+        _street = archive.decode("_street")
+        _subAdministrativeArea = archive.decode("_subAdministrativeArea")
+        _subLocality = archive.decode("_subLocality")
+
+        return CNPostalAddress(
+            _ISOCountryCode,
+            _city,
+            _country,
+            _postalCode,
+            _state,
+            _street,
+            _subAdministrativeArea,
+            _subLocality,
+        )
+
+
+# register the classes with bpylist.archiver
+archiver.update_class_map({"CNPostalAddress": CNPostalAddress})
+archiver.update_class_map(
+    {"PLRevGeoMapItemAdditionalPlaceInfo": PLRevGeoMapItemAdditionalPlaceInfo}
+)
+archiver.update_class_map({"PLRevGeoMapItem": PLRevGeoMapItem})
+archiver.update_class_map({"PLRevGeoLocationInfo": PLRevGeoLocationInfo})
+
+
+class PlaceInfo(ABC):
+    @property
+    @abstractmethod
+    def address_str(self):
+        pass
+
+    @property
+    @abstractmethod
+    def country_code(self):
+        pass
+
+    @property
+    @abstractmethod
+    def ishome(self):
+        pass
+
+    @property
+    @abstractmethod
+    def name(self):
+        pass
+
+    @property
+    @abstractmethod
+    def names(self):
+        pass
+
+    @property
+    @abstractmethod
+    def address(self):
+        pass
+
+
+class PlaceInfo4(PlaceInfo):
+    """ Reverse geolocation place info for a photo (Photos <= 4) """
+
+    def __init__(self, place_names, country_code):
+        """ place_names: list of place name tuples in ascending order by area 
+            tuple fields are: modelID, place name, place type, area, e.g.
+            [(5, "St James's Park", 45, 0), 
+            (4, 'Westminster', 16, 22097376), 
+            (3, 'London', 4, 1596146816), 
+            (2, 'England', 2, 180406091776), 
+            (1, 'United Kingdom', 1, 414681432064)] 
+            country_code: two letter country code for the country
+        """
+        self._place_names = place_names
+        self._country_code = country_code
+        self._process_place_info()
+
+    @property
+    def address_str(self):
+        return None
+
+    @property
+    def country_code(self):
+        return self._country_code
+
+    @property
+    def ishome(self):
+        return None
+
+    @property
+    def name(self):
+        return self._name
+
+    @property
+    def names(self):
+        return self._names
+
+    @property
+    def address(self):
+        return PostalAddress(None, None, None, None, None, None, None, None)
+
+    def __eq__(self, other):
+        if not isinstance(other, type(self)):
+            return False
+        else:
+            return (
+                self._place_names == other._place_names
+                and self._country_code == other._country_code
+            )
+
+    def _process_place_info(self):
+        """ Process place_names to set self._name and self._names """
+        places = self._place_names
+
+        # build a dictionary where key is placetype
+        places_dict = {}
+        for p in places:
+            # places in format:
+            # [(5, "St James's Park", 45, 0), ]
+            #   0: modelID
+            #   1: name
+            #   2: type
+            #   3: area
+            try:
+                places_dict[p[2]].append((p[1], p[3]))
+            except KeyError:
+                places_dict[p[2]] = [(p[1], p[3])]
+
+        # build list to populate PlaceNames tuple
+        # initialize with empty lists for each field in PlaceNames
+        place_info = [[]] * 19
+
+        # add the place names sorted by area (ascending)
+        # in Photos <=4, possible place type values are:
+        # 45: areasOfInterest (The relevant areas of interest associated with the placemark.)
+        # 44: body of water (includes both inlandWater and ocean)
+        # 43: subLocality (Additional city-level information for the placemark.
+        # 16: locality (The city associated with the placemark.)
+        # 4: subAdministrativeArea (Additional administrative area information for the placemark.)
+        # 2: administrativeArea (The state or province associated with the placemark.)
+        # 1: country
+        # mapping = mapping from PlaceNames to field in places_dict
+        # PlaceNames fields map to the placeType value in Photos5 (0..17)
+        # but place type in Photos <=4 has different values
+        # hence (3, 4) means PlaceNames[3] = places_dict[4] (sub_administrative_area)
+        mapping = [(1, 1), (2, 2), (3, 4), (4, 16), (18, 44), (8, 45)]
+        for field5, field4 in mapping:
+            try:
+                place_info[field5] = [
+                    p[0]
+                    for p in sorted(places_dict[field4], key=lambda place: place[1])
+                ]
+            except KeyError:
+                pass
+
+        place_names = PlaceNames(*place_info)
+        self._names = place_names
+
+        # build the name as it appears in Photos
+        # the length of the name is at most 3 fields and appears to be based on available
+        # reverse geolocation data in the following order (left to right, joined by ',')
+        # always has country if available then either area of interest and city OR
+        # city and state
+        # e.g. 4, 2, 1 OR 8, 4, 1
+        # 8 (45): area_of_interest
+        # 4 (16): locality / city
+        # 2 (2): administrative area (state/province)
+        # 1 (1): country
+        name_list = []
+        if place_names[8]:
+            name_list.append(place_names[8][0])
+            if place_names[4]:
+                name_list.append(place_names[4][0])
+        elif place_names[4]:
+            name_list.append(place_names[4][0])
+            if place_names[2]:
+                name_list.append(place_names[2][0])
+        elif place_names[2]:
+            name_list.append(place_names[2][0])
+
+        # add country
+        if place_names[1]:
+            name_list.append(place_names[1][0])
+
+        name = ", ".join(name_list)
+        self._name = name if name != "" else None
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __str__(self):
+        info = {
+            "name": self.name,
+            "names": self.names,
+            "country_code": self.country_code,
+        }
+        strval = "PlaceInfo(" + ", ".join([f"{k}='{v}'" for k, v in info.items()]) + ")"
+        return strval
+
+
+class PlaceInfo5(PlaceInfo):
+    """ Reverse geolocation place info for a photo (Photos >= 5) """
+
+    def __init__(self, revgeoloc_bplist):
+        """ revgeoloc_bplist: a binary plist blob containing 
+                a serialized PLRevGeoLocationInfo object """
+        self._bplist = revgeoloc_bplist
+        # todo: check for None?
+        self._plrevgeoloc = archiver.unarchive(revgeoloc_bplist)
+        self._process_place_info()
+
+    @property
+    def address_str(self):
+        """ returns the postal address as a string """
+        return self._plrevgeoloc.addressString
+
+    @property
+    def country_code(self):
+        """ returns the country code """
+        return self._plrevgeoloc.countryCode
+
+    @property
+    def ishome(self):
+        """ returns True if place is user's home address """
+        return self._plrevgeoloc.isHome
+
+    @property
+    def name(self):
+        """ returns local place name """
+        return self._name
+
+    @property
+    def names(self):
+        """ returns PlaceNames tuple with detailed reverse geolocation place names """
+        return self._names
+
+    @property
+    def address(self):
+        addr = self._plrevgeoloc.postalAddress
+        address = PostalAddress(
+            street=addr._street,
+            sub_locality=addr._subLocality,
+            city=addr._city,
+            sub_administrative_area=addr._subAdministrativeArea,
+            state_province=addr._state,
+            postal_code=addr._postalCode,
+            country=addr._country,
+            iso_country_code=addr._ISOCountryCode,
+        )
+        return address
+
+    def _process_place_info(self):
+        """ Process sortedPlaceInfos to set self._name and self._names """
+        places = self._plrevgeoloc.mapItem.sortedPlaceInfos
+
+        # build a dictionary where key is placetype
+        places_dict = {}
+        for p in places:
+            try:
+                places_dict[p.placeType].append((p.name, p.area))
+            except KeyError:
+                places_dict[p.placeType] = [(p.name, p.area)]
+
+        # build list to populate PlaceNames tuple
+        place_info = []
+        for field in range(18):
+            try:
+                # add the place names sorted by area (ascending)
+                place_info.append(
+                    [
+                        p[0]
+                        for p in sorted(places_dict[field], key=lambda place: place[1])
+                    ]
+                )
+            except:
+                place_info.append([])
+
+        # fill in body_of_water for compatibility with Photos <= 4
+        place_info.append(place_info[7] + place_info[9])
+
+        place_names = PlaceNames(*place_info)
+        self._names = place_names
+
+        # build the name as it appears in Photos
+        # the length of the name is variable and appears to be based on available
+        # reverse geolocation data in the following order (left to right, joined by ',')
+        # 8: area_of_interest
+        # 11: region (I've only seen this applied to islands)
+        # 4: locality / city
+        # 2: administrative area (state/province)
+        # 1: country
+        # 9: inland_water
+        # 7: ocean
+        name = ", ".join(
+            [
+                p[0]
+                for p in [
+                    place_names[8],  # area of interest
+                    place_names[11],  # region (I've only seen this applied to islands)
+                    place_names[4],  # locality / city
+                    place_names[2],  # administrative area (state/province)
+                    place_names[1],  # country
+                    place_names[9],  # inland_water
+                    place_names[7],  # ocean
+                ]
+                if p and p[0]
+            ]
+        )
+        self._name = name if name != "" else None
+
+    def __eq__(self, other):
+        if not isinstance(other, type(self)):
+            return False
+        else:
+            return self._plrevgeoloc == other._plrevgeoloc
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __str__(self):
+        info = {
+            "name": self.name,
+            "names": self.names,
+            "country_code": self.country_code,
+            "ishome": self.ishome,
+            "address_str": self.address_str,
+            "address": str(self.address),
+        }
+        strval = "PlaceInfo(" + ", ".join([f"{k}='{v}'" for k, v in info.items()]) + ")"
+        return strval

osxphotos/template.py

@@ -0,0 +1,64 @@
+""" Custom template system for osxphotos (implemented in PhotoInfo.render_template) """
+
+# Rolled my own template system because:
+# 1. Needed to handle multiple values (e.g. album, keyword)
+# 2. Needed to handle default values if template not found
+# 3. Didn't want user to need to know python (e.g. by using Mako which is
+#    already used elsewhere in this project)
+# 4. Couldn't figure out how to do #1 and #2 with str.format()
+#
+# This code isn't elegant but it seems to work well.  PRs gladly accepted.
+
+import locale
+
+# ensure locale set to user's locale
+locale.setlocale(locale.LC_ALL, "")
+
+# Permitted substitutions (each of these returns a single value or None)
+TEMPLATE_SUBSTITUTIONS = {
+    "{name}": "Current filename of the photo",
+    "{original_name}": "Photo's original filename when imported to Photos",
+    "{title}": "Title of the photo",
+    "{descr}": "Description of the photo",
+    "{created.date}": "Photo's creation date in ISO format, e.g. '2020-03-22'",
+    "{created.year}": "4-digit year of file creation time",
+    "{created.yy}": "2-digit year of file creation time",
+    "{created.mm}": "2-digit month of the file creation time (zero padded)",
+    "{created.month}": "Month name in user's locale of the file creation time",
+    "{created.mon}": "Month abbreviation in the user's locale of the file creation time",
+    "{created.doy}": "3-digit day of year (e.g Julian day) of file creation time, starting from 1 (zero padded)",
+    "{modified.date}": "Photo's modification date in ISO format, e.g. '2020-03-22'",
+    "{modified.year}": "4-digit year of file modification time",
+    "{modified.yy}": "2-digit year of file modification time",
+    "{modified.mm}": "2-digit month of the file modification time (zero padded)",
+    "{modified.month}": "Month name in user's locale of the file modification time",
+    "{modified.mon}": "Month abbreviation in the user's locale of the file modification time",
+    "{modified.doy}": "3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)",
+    "{place.name}": "Place name from the photo's reverse geolocation data, as displayed in Photos",
+    "{place.country_code}": "The ISO country code from the photo's reverse geolocation data",
+    "{place.name.country}": "Country name from the photo's reverse geolocation data",
+    "{place.name.state_province}": "State or province name from the photo's reverse geolocation data",
+    "{place.name.city}": "City or locality name from the photo's reverse geolocation data",
+    "{place.name.area_of_interest}": "Area of interest name (e.g. landmark or public place) from the photo's reverse geolocation data",
+    "{place.address}": "Postal address from the photo's reverse geolocation data, e.g. '2007 18th St NW, Washington, DC 20009, United States'",
+    "{place.address.street}": "Street part of the postal address, e.g. '2007 18th St NW'",
+    "{place.address.city}": "City part of the postal address, e.g. 'Washington'",
+    "{place.address.state_province}": "State/province part of the postal address, e.g. 'DC'",
+    "{place.address.postal_code}": "Postal code part of the postal address, e.g. '20009'",
+    "{place.address.country}": "Country name of the postal address, e.g. 'United States'",
+    "{place.address.country_code}": "ISO country code of the postal address, e.g. 'US'",
+}
+
+# Permitted multi-value substitutions (each of these returns None or 1 or more values)
+TEMPLATE_SUBSTITUTIONS_MULTI_VALUED = {
+    "{album}": "Album(s) photo is contained in",
+    "{folder_album}": "Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder",
+    "{keyword}": "Keyword(s) assigned to photo",
+    "{person}": "Person(s) / face(s) in a photo",
+}
+
+# Just the multi-valued substitution names without the braces
+MULTI_VALUE_SUBSTITUTIONS = [
+    field.replace("{", "").replace("}", "")
+    for field in TEMPLATE_SUBSTITUTIONS_MULTI_VALUED.keys()
+]

osxphotos/templates/xmp_sidecar.mako

@@ -0,0 +1,99 @@
+<!-- Created with osxphotos https://github.com/RhetTbull/osxphotos -->
+
+<%def name="dc_description(desc)">
+    % if desc is None:
+        <dc:description></dc:description>
+    % else:
+        <dc:description>${desc}</dc:description>
+    % endif
+</%def>
+
+<%def name="dc_title(title)">
+    % if title is None:
+        <dc:title></dc:title>
+    % else:
+        <dc:title>${title}</dc:title>
+    % endif
+</%def>
+
+<%def name="dc_subject(subject)">
+    % if subject:
+        <!-- keywords and persons listed in <dc:subject> as Photos does -->
+        <dc:subject>
+            <rdf:Seq>
+            % for subj in subject:
+                <rdf:li>${subj}</rdf:li>
+            % endfor
+            </rdf:Seq>
+        </dc:subject>
+    % endif
+</%def>
+
+<%def name="dc_datecreated(date)">
+    % if date is not None:
+        <photoshop:DateCreated>${date.isoformat()}</photoshop:DateCreated>
+    % endif
+</%def>
+
+<%def name="iptc_personinimage(persons)">
+    % if persons:
+        <Iptc4xmpExt:PersonInImage>
+            <rdf:Bag>
+                % for person in persons:
+                    <rdf:li>${person}</rdf:li>
+                % endfor
+            </rdf:Bag>
+        </Iptc4xmpExt:PersonInImage>
+    % endif 
+</%def>
+
+<%def name="dk_tagslist(keywords)">
+    % if keywords:
+        <digiKam:TagsList>
+            <rdf:Seq>
+            % for keyword in keywords:
+                <rdf:li>${keyword}</rdf:li>
+            % endfor
+            </rdf:Seq>
+        </digiKam:TagsList>
+    % endif
+</%def>
+
+<%def name="adobe_createdate(date)">
+    % if date is not None:
+        <xmp:CreateDate>${date.strftime("%Y-%m-%dT%H:%M:%S")}</xmp:CreateDate>
+    % endif
+</%def>
+
+<%def name="adobe_modifydate(date)">
+    % if date is not None:
+        <xmp:ModifyDate>${date.strftime("%Y-%m-%dT%H:%M:%S")}</xmp:ModifyDate>
+    % endif
+</%def>
+
+<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
+    <!-- mirrors Photos 5 "Export IPTC as XMP" option -->
+    <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+        <rdf:Description rdf:about="" 
+            xmlns:dc="http://purl.org/dc/elements/1.1/" 
+            xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
+            ${dc_description(photo.description)}
+            ${dc_title(photo.title)}
+            ${dc_subject(subjects)}
+            ${dc_datecreated(photo.date)}
+        </rdf:Description>
+        <rdf:Description rdf:about='' 
+            xmlns:Iptc4xmpExt='http://iptc.org/std/Iptc4xmpExt/2008-02-29/'>
+            ${iptc_personinimage(persons)}
+        </rdf:Description>
+        <rdf:Description rdf:about='' 
+            xmlns:digiKam='http://www.digikam.org/ns/1.0/'>
+            ${dk_tagslist(keywords)}
+        </rdf:Description>
+        <rdf:Description rdf:about='' 
+            xmlns:xmp='http://ns.adobe.com/xap/1.0/'>
+            ${adobe_createdate(photo.date)}
+            ${adobe_modifydate(photo.date)}
+        </rdf:Description>
+   </rdf:RDF>
+</x:xmpmeta>

osxphotos/utils.py

@@ -1,16 +1,25 @@
+import fnmatch
 import glob
 import logging
+import os
 import os.path
+import pathlib
 import platform
+import re
+import sqlite3
 import subprocess
+import sys
+import tempfile
 import urllib.parse
-from pathlib import Path
 from plistlib import load as plistload
 
 import CoreFoundation
+import CoreServices
 import objc
 from Foundation import *
 
+from osxphotos._applescript import AppleScript
+
 _DEBUG = False
 
 
@@ -109,6 +118,64 @@ def _dd_to_dms(dd):
     return int(deg_), int(min_), sec_
 
 
+def _hardlink_file(src, dest):
+    """ Hardlinks a file from src path to dest path 
+        src: source path as string 
+        dest: destination path as string
+        Raises exception if linking fails or either path is None """
+
+    if src is None or dest is None:
+        raise ValueError("src and dest must not be None", src, dest)
+
+    if not os.path.isfile(src):
+        raise FileNotFoundError("src file does not appear to exist", src)
+
+
+    # if error on copy, subprocess will raise CalledProcessError
+    try:
+        os.link(src, dest)
+    except Exception as e:
+        logging.critical(
+            f"ln returned error: {e.returncode} {e.stderr.decode(sys.getfilesystemencoding()).rstrip()}"
+        )
+        raise e
+
+    
+
+def _copy_file(src, dest, norsrc=False):
+    """ Copies a file from src path to dest path 
+        src: source path as string 
+        dest: destination path as string
+        norsrc: (bool) if True, uses --norsrc flag with ditto so it will not copy
+                resource fork or extended attributes.  May be useful on volumes that
+                don't work with extended attributes (likely only certain SMB mounts)
+                default is False
+        Uses ditto to perform copy; will silently overwrite dest if it exists
+        Raises exception if copy fails or either path is None """
+
+    if src is None or dest is None:
+        raise ValueError("src and dest must not be None", src, dest)
+
+    if not os.path.isfile(src):
+        raise FileNotFoundError("src file does not appear to exist", src)
+
+    if norsrc:
+        command = ["/usr/bin/ditto", "--norsrc", src, dest]
+    else:
+        command = ["/usr/bin/ditto", src, dest]
+
+    # if error on copy, subprocess will raise CalledProcessError
+    try:
+        result = subprocess.run(command, check=True, stderr=subprocess.PIPE)
+    except subprocess.CalledProcessError as e:
+        logging.critical(
+            f"ditto returned error: {e.returncode} {e.stderr.decode(sys.getfilesystemencoding()).rstrip()}"
+        )
+        raise e
+
+    return result.returncode
+
+
 def dd_to_dms_str(lat, lon):
     """ convert latitude, longitude in degrees to degrees, minutes, seconds as string """
     """ lat: latitude in degrees  """
@@ -141,15 +208,16 @@ def dd_to_dms_str(lat, lon):
 def get_system_library_path():
     """ return the path to the system Photos library as string """
     """ only works on MacOS 10.15+ """
-    """ on earlier versions, will raise exception """
+    """ on earlier versions, returns None """
     _, major, _ = _get_os_version()
     if int(major) < 15:
-        raise Exception(
-            "get_system_library_path not implemented for MacOS < 10.15", major
+        logging.debug(
+            f"get_system_library_path not implemented for MacOS < 10.15: you have {major}"
         )
+        return None
 
-    plist_file = Path(
-        str(Path.home())
+    plist_file = pathlib.Path(
+        str(pathlib.Path.home())
         + "/Library/Containers/com.apple.photolibraryd/Data/Library/Preferences/com.apple.photolibraryd.plist"
     )
     if plist_file.is_file():
@@ -171,9 +239,8 @@ def get_system_library_path():
 def get_last_library_path():
     """ returns the path to the last opened Photos library 
         If a library has never been opened, returns None """
-    # TODO: Need a module level method for this and another PhotosDB method to get current library path
-    plist_file = Path(
-        str(Path.home())
+    plist_file = pathlib.Path(
+        str(pathlib.Path.home())
         + "/Library/Containers/com.apple.Photos/Data/Library/Preferences/com.apple.Photos.plist"
     )
     if plist_file.is_file():
@@ -189,6 +256,8 @@ def get_last_library_path():
 
     if photosurlref is not None:
         # use CFURLCreateByResolvingBookmarkData to de-serialize bookmark data into a CFURLRef
+        # pylint: disable=no-member
+        # pylint: disable=undefined-variable
         photosurl = CoreFoundation.CFURLCreateByResolvingBookmarkData(
             kCFAllocatorDefault, photosurlref, 0, None, None, None, None
         )
@@ -226,7 +295,7 @@ def list_photo_libraries():
     # On older MacOS versions, mdfind appears to ignore some libraries
     # glob to find libraries in ~/Pictures then mdfind to find all the others
     # TODO: make this more robust
-    lib_list = glob.glob(f"{str(Path.home())}/Pictures/*.photoslibrary")
+    lib_list = glob.glob(f"{str(pathlib.Path.home())}/Pictures/*.photoslibrary")
 
     # On older OS, may not get all libraries so make sure we get the last one
     last_lib = get_last_library_path()
@@ -259,3 +328,191 @@ def create_path_by_date(dest, dt):
     if not os.path.isdir(new_dest):
         os.makedirs(new_dest)
     return new_dest
+
+
+def get_preferred_uti_extension(uti):
+    """ get preferred extension for a UTI type
+        uti: UTI str, e.g. 'public.jpeg'
+        returns: preferred extension as str """
+
+    # reference: https://developer.apple.com/documentation/coreservices/1442744-uttypecopypreferredtagwithclass?language=objc
+
+    ext = CoreServices.UTTypeCopyPreferredTagWithClass(
+        uti, CoreServices.kUTTagClassFilenameExtension
+    )
+    return ext
+
+
+def findfiles(pattern, path_):
+    """Returns list of filenames from path_ matched by pattern
+       shell pattern. Matching is case-insensitive."""
+    # See: https://gist.github.com/techtonik/5694830
+
+    rule = re.compile(fnmatch.translate(pattern), re.IGNORECASE)
+    return [name for name in os.listdir(path_) if rule.match(name)]
+
+
+# TODO: this doesn't always work, still looking for a way to
+# force Photos to open the library being operated on
+# def _open_photos_library_applescript(library_path):
+#     """ Force Photos to open a specific library
+#         library_path: path to the Photos library """
+#     open_scpt = AppleScript(
+#         f"""
+#         on openLibrary
+#             tell application "Photos"
+# 	          open POSIX file "{library_path}"
+#             end tell
+#         end openLibrary
+#         """
+#     )
+#     open_scpt.run()
+
+
+def _export_photo_uuid_applescript(
+    uuid,
+    dest,
+    filestem=None,
+    original=True,
+    edited=False,
+    live_photo=False,
+    timeout=120,
+    burst=False,
+):
+    """ Export photo to dest path using applescript to control Photos
+        If photo is a live photo, exports both the photo and associated .mov file
+        uuid: UUID of photo to export
+        dest: destination path to export to
+        filestem: (string) if provided, exported filename will be named stem.ext 
+                  where ext is extension of the file exported by photos (e.g. .jpeg, .mov, etc)
+                  If not provided, file will be named with whatever name Photos uses
+                  If filestem.ext exists, it wil be overwritten
+        original: (boolean) if True, export original image; default = True
+        edited: (boolean) if True, export edited photo; default = False
+                If photo not edited and edited=True, will still export the original image
+                caller must verify image has been edited
+        *Note*: must be called with either edited or original but not both, 
+                will raise error if called with both edited and original = True
+        live_photo: (boolean) if True, export associated .mov live photo; default = False
+        timeout: timeout value in seconds; export will fail if applescript run time exceeds timeout
+        burst: (boolean) set to True if file is a burst image to avoid Photos export error
+        Returns: list of paths to exported file(s) or None if export failed
+        Note: For Live Photos, if edited=True, will export a jpeg but not the movie, even if photo
+              has not been edited. This is due to how Photos Applescript interface works.
+    """
+
+    # setup the applescript to do the export
+    export_scpt = AppleScript(
+        """ 
+		on export_by_uuid(theUUID, thePath, original, edited, theTimeOut)
+			tell application "Photos"
+				set thePath to thePath
+				set theItem to media item id theUUID
+				set theFilename to filename of theItem
+				set itemList to {theItem}
+				
+				if original then
+					with timeout of theTimeOut seconds
+						export itemList to POSIX file thePath with using originals
+					end timeout
+				end if
+				
+				if edited then
+					with timeout of theTimeOut seconds
+						export itemList to POSIX file thePath
+					end timeout
+				end if
+				
+				return theFilename
+			end tell
+
+		end export_by_uuid
+		"""
+    )
+
+    dest = pathlib.Path(dest)
+    if not dest.is_dir:
+        raise ValueError(f"dest {dest} must be a directory")
+
+    if not original ^ edited:
+        raise ValueError(f"edited or original must be True but not both")
+
+    tmpdir = tempfile.TemporaryDirectory(prefix="osxphotos_")
+
+    # export original
+    filename = None
+    try:
+        filename = export_scpt.call(
+            "export_by_uuid", uuid, tmpdir.name, original, edited, timeout
+        )
+    except Exception as e:
+        logging.warning("Error exporting uuid %s: %s" % (uuid, str(e)))
+        return None
+
+    if filename is not None:
+        # need to find actual filename as sometimes Photos renames JPG to jpeg on export
+        # may be more than one file exported (e.g. if Live Photo, Photos exports both .jpeg and .mov)
+        # TemporaryDirectory will cleanup on return
+        filename_stem = pathlib.Path(filename).stem
+        files = glob.glob(os.path.join(tmpdir.name, "*"))
+        exported_paths = []
+        for fname in files:
+            path = pathlib.Path(fname)
+            if len(files) > 1 and not live_photo and path.suffix.lower() == ".mov":
+                # it's the .mov part of live photo but not requested, so don't export
+                logging.debug(f"Skipping live photo file {path}")
+                continue
+            if len(files) > 1 and burst and path.stem != filename_stem:
+                # skip any burst photo that's not the one we asked for
+                logging.debug(f"Skipping burst photo file {path}")
+                continue
+            if filestem:
+                # rename the file based on filestem, keeping original extension
+                dest_new = dest / f"{filestem}{path.suffix}"
+            else:
+                # use the name Photos provided
+                dest_new = dest / path.name
+            logging.debug(f"exporting {path} to dest_new: {dest_new}")
+            _copy_file(str(path), str(dest_new))
+            exported_paths.append(str(dest_new))
+        return exported_paths
+    else:
+        return None
+
+
+def _open_sql_file(dbname):
+    """ opens sqlite file dbname in read-only mode
+        returns tuple of (connection, cursor) """
+    try:
+        dbpath = pathlib.Path(dbname).resolve()
+        conn = sqlite3.connect(f"{dbpath.as_uri()}?mode=ro", timeout=1, uri=True)
+        c = conn.cursor()
+    except sqlite3.Error as e:
+        sys.exit(f"An error occurred opening sqlite file: {e.args[0]} {dbname}")
+    return (conn, c)
+
+
+def _db_is_locked(dbname):
+    """ check to see if a sqlite3 db is locked
+        returns True if database is locked, otherwise False
+        dbname: name of database to test """
+
+    # first, check to see if lock file exists, if so, assume the file is locked
+    lock_name = f"{dbname}.lock"
+    if os.path.exists(lock_name):
+        logging.debug(f"{dbname} is locked")
+        return True
+
+    # no lock file so try to read from the database to see if it's locked
+    locked = None
+    try:
+        (conn, c) = _open_sql_file(dbname)
+        c.execute("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name;")
+        conn.close()
+        logging.debug(f"{dbname} is not locked")
+        locked = False
+    except:
+        logging.debug(f"{dbname} is locked")
+        locked = True
+
+    return locked

requirements.txt

@@ -1,23 +1,36 @@
+altgraph==0.17
 ansimarkup==1.4.0
+appdirs==1.4.3
 astroid==2.2.5
 atomicwrites==1.3.0
 attrs==19.1.0
 better-exceptions-fork==0.2.1.post6
+# bpylist2==2.0.3;python_version<"3.8"
+https://github.com/RhetTbull/bpylist/releases/download/v2.0.3/bpylist2-2.0.3.tar.gz#egg=bpylist2;python_version<"3.8"
+bpylist2==3.0.0;python_version>="3.8"
 certifi==2019.3.9
 Click==7.0
 colorama==0.4.1
 coverage==4.5.4
-importlib-metadata==0.18
+importlib-metadata>=0.18
 isort==4.3.20
 lazy-object-proxy==1.4.1
 loguru==0.2.5
+macholib==1.14
+Mako==1.1.1
+MarkupSafe==1.1.1
 mccabe==0.6.1
+modulegraph==0.18
 more-itertools==7.2.0
--e git+https://github.com/RhetTbull/osxphotos.git@0271b8ad9daf8b2fb80ce81e894478370e421379#egg=osxphotos
 packaging==19.0
+pathspec==0.7.0
+pathvalidate==2.2.1
 pluggy==0.12.0
 py==1.8.0
+py2app==0.21
 Pygments==2.4.2
+PyInstaller==3.6
+pyinstaller-setuptools==2019.3
 pylint==2.3.1
 pyobjc==6.0.1
 pyobjc-core==6.0.1
@@ -134,12 +147,12 @@ pyobjc-framework-VideoToolbox==6.0.1
 pyobjc-framework-Vision==6.0.1
 pyobjc-framework-WebKit==6.0.1
 pyparsing==2.4.1.1
-pytest==5.3.1
-pytest-cov==2.8.1
-pytest-sugar==0.9.2
 PyYAML==5.1.2
+regex==2020.2.20
 six==1.12.0
 termcolor==1.1.0
+toml==0.10.0
+typed-ast==1.4.1
 wcwidth==0.1.7
 wrapt==1.11.1
 zipp==0.5.2

setup.py

@@ -3,7 +3,7 @@
 #
 # setup.py script for osxphotos
 #
-# Copyright (c) 2019 Rhet Turnbull, rturnbull+git@gmail.com
+# Copyright (c) 2019, 2020 Rhet Turnbull, rturnbull+git@gmail.com
 # All rights reserved.
 #
 # Permission is hereby granted, free of charge, to any person
@@ -27,30 +27,50 @@
 # SOFTWARE.
 
 import os
+import platform
+
 from setuptools import find_packages, setup
 
-this_directory = os.path.abspath(os.path.dirname(__file__))
-with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
-    long_description = f.read()
+# python version as 2-digit float (e.g. 3.6)
+py_ver = float(".".join(platform.python_version_tuple()[:2]))
 
+# holds config info read from disk
 about = {}
+this_directory = os.path.abspath(os.path.dirname(__file__))
+
+# get version info from _version
 with open(
     os.path.join(this_directory, "osxphotos", "_version.py"), mode="r", encoding="utf-8"
 ) as f:
     exec(f.read(), about)
 
+# read README.md into long_description
+with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
+    about["long_description"] = f.read()
+
+# ugly hack to install custom version of bpylist2 needed for Python < 3.8
+# the stock version of bylist2==2.0.3 causes an error related to
+# "pkg_resources.ContextualVersionConflict: (pycodestyle 2.3.1..."
+# PEP 508 no help here as URL-based lookups not allowed in PyPI packages
+# if you know a better way, PRs welcome!
+# once I go to 3.8+ required, this won't be necessary as bpylist2 3.0+ solves this issue
+if py_ver < 3.8:
+    os.system(
+        "python3 -m pip install git+git://github.com/RhetTbull/bpylist2.git#egg=bpylist2"
+    )
+
 setup(
     name="osxphotos",
     version=about["__version__"],
     description="Manipulate (read-only) Apple's Photos app library on Mac OS X",
-    long_description=long_description,
+    long_description=about["long_description"],
     long_description_content_type="text/markdown",
     author="Rhet Turnbull",
     author_email="rturnbull+git@gmail.com",
     url="https://github.com/RhetTbull/",
     project_urls={"GitHub": "https://github.com/RhetTbull/osxphotos"},
     download_url="https://github.com/RhetTbull/osxphotos",
-    packages=find_packages(exclude=["tests", "examples"]),
+    packages=find_packages(exclude=["tests", "examples", "utils"]),
     license="License :: OSI Approved :: MIT License",
     classifiers=[
         "Development Status :: 4 - Beta",
@@ -58,9 +78,19 @@ setup(
         "Intended Audience :: Developers",
         "License :: OSI Approved :: MIT License",
         "Operating System :: MacOS :: MacOS X",
-        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.8",
         "Topic :: Software Development :: Libraries :: Python Modules",
     ],
-    install_requires=["pyobjc", "Click", "pyyaml"],
+    install_requires=[
+        "pyobjc>=6.0.1",
+        "Click>=7",
+        "PyYAML>=5.1.2",
+        "Mako>=1.1.1",
+        "bpylist2==2.0.3;python_version<'3.8'",
+        "bpylist2==3.0.0;python_version>='3.8'",
+        "pathvalidate==2.2.1",
+        "dataclasses==0.7;python_version<'3.7'",
+    ],
     entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]},
+    include_package_data=True,
 )

tests/README.md

@@ -1,14 +1,22 @@
 # Tests for osxphotos #
 
 ## Running Tests ##
-Tests require pytest:
+Tests require pytest and pytest-mock:
 `pip install pytest`
+`pip install pytest-mock`
 
 To run the tests, do the following from the main source folder:
 `python -m pytest tests/`
 
 Running the tests this way allows the library to be tested without installing it.
 
+## Skipped Tests ##
+A few tests will look for certain environment variables to determine if they should run.
+
+Some of the export tests rely on photos in my local library and will look for `OSXPHOTOS_TEST_EXPORT=1` to determine if they should run.
+
+One test for locale does not run on GitHub's automated workflow and will look for `OSXPHOTOS_TEST_LOCALE=1` to determine if it should be run.  If you want to run this test, set the environment variable.  
+
 ## Attribution ##
 These tests utilize a test Photos library. The test library is populated with photos from [flickr](https://www.flickr.com).  All images used are licensed under Creative Commons 2.0 Attribution [license](https://creativecommons.org/licenses/by/2.0/).  
 
@@ -17,5 +25,6 @@ Images used from:
 - [Carlos Montesdeoca](https://www.flickr.com/photos/carlosmontesdeocastudio)
 - [Rydale Clothing](https://www.flickr.com/photos/rydaleclothing)
 - [Marco Verch](https://www.flickr.com/photos/30478819@N08/48228222317/)
+- [K M](https://www.flickr.com/photos/153387643@N08/49334338022/)
 
 

tests/Test-10.12.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2019-12-08T16:44:38Z</date>
+		<date>2020-04-17T18:39:50Z</date>
 	</dict>
 	<key>PXPeopleScreenUnlocked</key>
 	<true/>

tests/Test-10.12.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2019-12-30T14:19:49Z</date>
+	<date>2020-04-17T18:40:46Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2019-12-30T19:26:32Z</date>
+	<date>2020-04-17T18:39:51Z</date>
 </dict>
 </plist>

tests/Test-10.12.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2019-12-28T22:33:47Z</date>
+	<date>2020-04-17T18:39:52Z</date>
 </dict>
 </plist>

tests/Test-10.12.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>414</integer>
+	<integer>502</integer>
 	<key>LibraryBuildTag</key>
 	<string>E3E46F2A-7168-4973-AB3E-5848F80BFC7D</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.12.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>414</integer>
+		<integer>502</integer>
 		<key>LibraryBuildTag</key>
 		<string>E3E46F2A-7168-4973-AB3E-5848F80BFC7D</string>
 		<key>LibrarySchemaVersion</key>

tests/Test-10.13.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -8,8 +8,11 @@
 		<array/>
 		<key>ExpandedSidebarItemIdentifiers</key>
 		<array>
+			<string>obfeGcvoT1auxoh2Tu86OQ</string>
 			<string>TopLevelAlbums</string>
 			<string>TopLevelSlideshows</string>
+			<string>MBS8+gBrQCWQxmcav+C8HQ</string>
+			<string>cHwwVoUiQ8a2nZNXgVsnCA</string>
 		</array>
 		<key>IPXWorkspaceControllerZoomLevelsKey</key>
 		<dict>
@@ -23,11 +26,11 @@
 			<key>key</key>
 			<integer>1</integer>
 			<key>lastKnownDisplayName</key>
-			<string>September 28, 2018</string>
+			<string>Pumpkin Farm (1)</string>
 			<key>type</key>
 			<string>album</string>
 			<key>uuid</key>
-			<string>+Ep8CrNRRhea9eVA618FMg</string>
+			<string>AU8Gp8bwRlOvngZFgwXBdg</string>
 		</dict>
 		<key>lastKnownItemCounts</key>
 		<dict>

tests/Test-10.13.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2019-07-28T01:23:52Z</date>
+	<date>2020-04-30T14:09:38Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2019-07-28T01:23:52Z</date>
+	<date>2020-05-01T04:27:48Z</date>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/private/com.apple.photoanalysisd/VisionService/PersonPromoter

@@ -2,7 +2,11 @@
 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0">
 <dict>
+	<key>ProcessedInQuiescentState</key>
+	<true/>
 	<key>SuggestedMeIdentifier</key>
 	<string></string>
+	<key>Version</key>
+	<integer>3</integer>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/private/com.apple.photomodel/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2019-07-27T12:01:15Z</date>
+		<date>2020-05-01T03:34:50Z</date>
 	</dict>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2019-07-26T20:15:18Z</date>
+	<date>2020-04-30T12:51:41Z</date>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>615</integer>
+	<integer>670</integer>
 	<key>LibraryBuildTag</key>
 	<string>BEA5F0E8-BA6B-4462-8F73-3E53BBE4C943</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.13.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>615</integer>
+		<integer>670</integer>
 		<key>LibraryBuildTag</key>
 		<string>BEA5F0E8-BA6B-4462-8F73-3E53BBE4C943</string>
 		<key>LibrarySchemaVersion</key>
@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-26T20:15:17Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2019-07-27T12:01:15Z</date>
+	<date>2020-05-01T03:34:50Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>