Improved handling of deleted photos, #332

doc: start with examples before the export reference, thanks to @synox

.all-contributorsrc

@@ -6,7 +6,8 @@
   "files": [
     "README.md"
   ],
-  "imageSize": 100,
+  "imageSize": 75,
+  "badgeTemplate": "[![All Contributors](https://img.shields.io/badge/all_contributors-<%= contributors.length %>-orange.svg?style=flat)](#contributors)",
   "commit": true,
   "commitConvention": "none",
   "contributors": [
@@ -118,6 +119,15 @@
       "contributions": [
         "doc"
       ]
+    },
+    {
+      "login": "synox",
+      "name": "Aravindo Wingeier",
+      "avatar_url": "https://avatars2.githubusercontent.com/u/2250964?v=4",
+      "profile": "https://github.com/synox",
+      "contributions": [
+        "doc"
+      ]
     }
   ],
   "contributorsPerLine": 7

CHANGELOG.md

@@ -4,7 +4,33 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
-#### [v0.39.5](https://github.com/RhetTbull/osxphotos/compare/v0.39.3...v0.39.5)
+#### [v0.39.7](https://github.com/RhetTbull/osxphotos/compare/v0.39.6...v0.39.7)
+
+> 3 January 2021
+
+- doc: start with examples before the export reference, thanks to @synox  [`#327`](https://github.com/RhetTbull/osxphotos/pull/327)
+- Added tag_groups arg to ExifTool.asdict(), issue #324 [`2480f2a`](https://github.com/RhetTbull/osxphotos/commit/2480f2a325dbb09689f8c417618b7b9e976bfcb9)
+- doc: start with examples before the export reference [`7c7bf1b`](https://github.com/RhetTbull/osxphotos/commit/7c7bf1be6b6382a995a4e17906adfd8720d0a1c3)
+- Updated dependencies in README.md [`b1cab32`](https://github.com/RhetTbull/osxphotos/commit/b1cab32ff4c7b65ae4c9a5a9a11c175dbd487c0a)
+- remove extra spaces [`a59bb5b`](https://github.com/RhetTbull/osxphotos/commit/a59bb5b02f10fa554dae346a7271be37f50d8bcc)
+- Adding back  dependency https://github.com/RhetTbull/PhotoScript) [`7c8bfc8`](https://github.com/RhetTbull/osxphotos/commit/7c8bfc811ab3a93dabadf1655f7d0e217d6c7b01)
+
+#### [v0.39.6](https://github.com/RhetTbull/osxphotos/compare/v0.39.5...v0.39.6)
+
+> 3 January 2021
+
+- Make readme easier for beginners, thanks to @synox  [`#326`](https://github.com/RhetTbull/osxphotos/pull/326)
+- doc simplify readme [`02ef0f9`](https://github.com/RhetTbull/osxphotos/commit/02ef0f9a254e83a3729a09cea1ae523407074896)
+- Added exception handling/capture for convert-to-jpeg, issue #322 [`05f111a`](https://github.com/RhetTbull/osxphotos/commit/05f111a287e882ed6b451a550a87753501316aba)
+-  Add @synox as a contributor [`83915c6`](https://github.com/RhetTbull/osxphotos/commit/83915c65abb880036f80ebd830eb1e34292f9599)
+
+#### [v0.39.5](https://github.com/RhetTbull/osxphotos/compare/v0.39.4...v0.39.5)
+
+> 3 January 2021
+
+- Cleanup up the readme [`38842ff`](https://github.com/RhetTbull/osxphotos/commit/38842ff9249e6f5b3069a88a759c8df97ddce51c)
+
+#### [v0.39.4](https://github.com/RhetTbull/osxphotos/compare/v0.39.3...v0.39.4)
 
 > 3 January 2021
 

README.md

@@ -3,7 +3,7 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Python package](https://github.com/RhetTbull/osxphotos/workflows/Python%20package/badge.svg)](https://github.com/RhetTbull/osxphotos/workflows/Python%20package/badge.svg)
 <!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
-[![All Contributors](https://img.shields.io/badge/all_contributors-12-orange.svg?style=flat-square)](#contributors-)
+[![All Contributors](https://img.shields.io/badge/all_contributors-13-orange.svg?style=flat)](#contributors)
 <!-- ALL-CONTRIBUTORS-BADGE:END -->
 
 - [OSXPhotos](#osxphotos)
@@ -11,7 +11,8 @@
   * [Supported operating systems](#supported-operating-systems)
   * [Installation instructions](#installation-instructions)
   * [Command Line Usage](#command-line-usage)
-  * [Example uses of the package](#example-uses-of-the-package)
+    + [Command line examples](#command-line-examples)
+    + [Command line reference: export](#command-line-reference-export)
   * [Package Interface](#package-interface)
     + [PhotosDB](#photosdb)
     + [PhotoInfo](#photoinfo)
@@ -40,31 +41,29 @@
   
 ## What is osxphotos?
 
-OSXPhotos provides the ability to interact with and query Apple's Photos.app library database on MacOS. Using this package you can query the Photos database for information about the photos stored in a Photos library on your Mac--for example, file name, file path, and metadata such as keywords/tags, persons/faces, albums, etc. You can also easily export both the original and edited photos.
+OSXPhotos provides the ability to interact with and query Apple's Photos.app library on macOS. You can query the Photos library database -- for example, file name, file path, and metadata such as keywords/tags, persons/faces, albums, etc. You can also easily export both the original and edited photos.
 
 ## Supported operating systems
 
-Only works on MacOS (aka Mac OS X). Tested on MacOS 10.12.6 / Photos 2.0, 10.13.6 / Photos 3.0, MacOS 10.14.5, 10.14.6 / Photos 4.0, MacOS 10.15.1 - 10.15.7 / Photos 5.0.
+Only works on macOS (aka Mac OS X). Tested on macOS Sierra (10.12.6) until macOS Catalina (10.15.7). 
 
-Beta support for MacOS 10.16/MacOS 11 Big Sur Beta / Photos 6.0.  Not tested on M1 / Apple silicon Macs.
+| macOS Version     | macOS name | Photos.app version |
+| ----------------- |------------|:-------------------|
+| 10.16             | Big Sur    | 6.0 ⚠️ Beta support, not tested on M1 / Apple silicon Macs. |
+| 10.15.1 - 10.15.7 | Catalina   | 5.0 ✅             |
+| 10.14.5, 10.14.6  | Mojave     | 4.0 ✅             |
+| 10.13.6           | High Sierra| 3.0 ✅             |
+| 10.12.6           | Sierra     | 2.0 ✅             |
 
-Requires python >= 3.7. 
+This package will read Photos databases for any supported version on any supported macOS version.  E.g. you can read a database created with Photos 5.0 on MacOS 10.15 on a machine running macOS 10.12 and vice versa.
 
-This package will read Photos databases for any supported version on any supported OS version.  E.g. you can read a database created with Photos 5.0 on MacOS 10.15 on a machine running MacOS 10.12 and vice versa.
+Requires python >= `3.7`. 
 
 
-## Installation instructions
-
-OSXPhotos uses setuptools, thus simply run:
-
-	python3 setup.py install
-
-You can also install directly from [pypi](https://pypi.org/project/osxphotos/):
-
-    pip install osxphotos
-
-I recommend you create a [virtual environment](https://docs.python.org/3/tutorial/venv.html) before installing osxphotos.
+## Installation 
+If you are new to python, I recommend you to install using pipx. See other advanced options below. 
 
+### Installation using pipx
 If you aren't familiar with installing python applications, I recommend you install `osxphotos` with [pipx](https://github.com/pipxproject/pipx). If you use `pipx`, you will not need to create a virtual environment as `pipx` takes care of this. The easiest way to do this on a Mac is to use [homebrew](https://brew.sh/):
 
 - Open `Terminal` (search for `Terminal` in Spotlight or look in `Applications/Utilities`)
@@ -73,13 +72,26 @@ If you aren't familiar with installing python applications, I recommend you inst
 - Then type this: `pipx install osxphotos`
 - Now you should be able to run `osxphotos` by typing: `osxphotos`
 
-**WARNING** The git repo for this project is very large (> 1GB) because it contains multiple Photos libraries used for testing on different versions of MacOS.  If you just want to use the osxphotos package in your own code, I recommend you install the latest version from [PyPI](https://pypi.org/project/osxphotos/) which does not include all the test libraries. If you just want to use the command line utility, you can download a pre-built executable of the latest [release](https://github.com/RhetTbull/osxphotos/releases) or you can install via `pip` which also installs the command line app.  If you aren't comfortable with running python on your Mac, start with the pre-built executable or `pipx` as described above.
+### Installation using pip
+You can also install directly from [pypi](https://pypi.org/project/osxphotos/):
+
+    pip install osxphotos
+
+### Installation from git repository
+OSXPhotos uses setuptools, thus simply run:
+
+	git clone https://github.com/RhetTbull/osxphotos.git
+	cd osxphotos
+	python3 setup.py install
+
+I recommend you create a [virtual environment](https://docs.python.org/3/tutorial/venv.html) before installing osxphotos.
+
+**WARNING** The git repo for this project is very large (> 1GB) because it contains multiple Photos libraries used for testing on different versions of macOS.  If you just want to use the osxphotos package in your own code, I recommend you install the latest version from [PyPI](https://pypi.org/project/osxphotos/) which does not include all the test libraries. If you just want to use the command line utility, you can download a pre-built executable of the latest [release](https://github.com/RhetTbull/osxphotos/releases) or you can install via `pip` which also installs the command line app.  If you aren't comfortable with running python on your Mac, start with the pre-built executable or `pipx` as described above.
 
 ## Command Line Usage
 
 This package will install a command line utility called `osxphotos` that allows you to query the Photos database.  Alternatively, you can also run the command line utility like this: `python3 -m osxphotos`
 
-After installing per instructions above, you should be able to run `osxphotos` on the command line:
 
 ```
 > osxphotos
@@ -114,7 +126,37 @@ Commands:
 
 To get help on a specific command, use `osxphotos help <command_name>`
 
-Example: `osxphotos help export`
+### Command line examples
+
+#### export all photos to ~/Desktop/export group in folders by date created
+
+`osxphotos export --export-by-date ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
+
+**Note**: Photos library/database path can also be specified using `--db` option:
+
+`osxphotos export --export-by-date --db ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
+
+#### find all photos with keyword "Kids" and output results to json file named results.json:
+
+`osxphotos query --keyword Kids --json ~/Pictures/Photos\ Library.photoslibrary >results.json`
+
+#### export photos to file structure based on 4-digit year and full name of month of photo's creation date:
+
+`osxphotos export ~/Desktop/export --directory "{created.year}/{created.month}"`
+
+(by default, it will attempt to use the system library)
+
+#### export photos to file structure based on 4-digit year of photo's creation date and add keywords for media type and labels (labels are only awailable on Photos 5 and higher):
+
+`osxphotos export ~/Desktop/export --directory "{created.year}" --keyword-template "{label}" --keyword-template "{media_type}"` 
+
+#### export default library using 'country name/year' as output directory (but use "NoCountry/year" if country not specified), add persons, album names, and year as keywords, write exif metadata to files when exporting, update only changed files, print verbose ouput
+
+`osxphotos export ~/Desktop/export --directory "{place.name.country,NoCountry}/{created.year}"  --person-keyword --album-keyword --keyword-template "{created.year}" --exiftool --update --verbose`
+
+### Command line reference: export
+
+`osxphotos help export`
 
 ```
 Usage: osxphotos export [OPTIONS] [PHOTOS_LIBRARY]... DEST
@@ -949,29 +991,7 @@ Substitution              Description
                           algorithms).
 ```
 
-Example: export all photos to ~/Desktop/export group in folders by date created
 
-`osxphotos export --export-by-date ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
-
-**Note**: Photos library/database path can also be specified using --db option:
-
-`osxphotos export --export-by-date --db ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
-
-Example: find all photos with keyword "Kids" and output results to json file named results.json:
-
-`osxphotos query --keyword Kids --json ~/Pictures/Photos\ Library.photoslibrary >results.json`
-
-Example: export photos to file structure based on 4-digit year and full name of month of photo's creation date:
-
-`osxphotos export ~/Desktop/export --directory "{created.year}/{created.month}"`
-
-Example: export photos to file structure based on 4-digit year of photo's creation date and add keywords for media type and labels (labels are only awailable on Photos 5 and higher):
-
-`osxphotos export ~/Desktop/export --directory "{created.year}" --keyword-template "{label}" --keyword-template "{media_type}"` 
-
-Example: export default library using 'country name/year' as output directory (but use "NoCountry/year" if country not specified), add persons, album names, and year as keywords, write exif metadata to files when exporting, update only changed files, print verbose ouput
-
-`osxphotos export ~/Desktop/export --directory "{place.name.country,NoCountry}/{created.year}"  --person-keyword --album-keyword --keyword-template "{created.year}" --exiftool --update --verbose`
 
 
 ## Example uses of the package 
@@ -1718,7 +1738,7 @@ exiftool must be installed in the path for this to work.  If exiftool cannot be
 
 `ExifTool` provides the following methods:
 
-- `asdict()`: returns all EXIF metadata found in the file as a dictionary in following form (Note: this shows just a subset of available metadata).  See [exiftool](https://exiftool.org/) documentation to understand which metadata keys are available.
+- `asdict(tag_groups=True)`: returns all EXIF metadata found in the file as a dictionary in following form (Note: this shows just a subset of available metadata).  See [exiftool](https://exiftool.org/) documentation to understand which metadata keys are available. If `tag_groups` is True (default) dict keys are in form "GROUP:TAG", e.g. "IPTC:Keywords". If `tag_groups` is False, dict keys do not have group names, e.g. "Keywords".
 
 ```python
 {'Composite:Aperture': 2.2,
@@ -2547,20 +2567,21 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
 <!-- markdownlint-disable -->
 <table>
   <tr>
-    <td align="center"><a href="https://github.com/britiscurious"><img src="https://avatars1.githubusercontent.com/u/25646439?v=4?s=100" width="100px;" alt=""/><br /><sub><b>britiscurious</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=britiscurious" title="Documentation">📖</a> <a href="https://github.com/RhetTbull/osxphotos/commits?author=britiscurious" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/mwort"><img src="https://avatars3.githubusercontent.com/u/8170417?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Michel Wortmann</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=mwort" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/PabloKohan"><img src="https://avatars3.githubusercontent.com/u/8790976?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Pablo 'merKur' Kohan</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=PabloKohan" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/hshore29"><img src="https://avatars2.githubusercontent.com/u/7023497?v=4?s=100" width="100px;" alt=""/><br /><sub><b>hshore29</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=hshore29" title="Code">💻</a></td>
-    <td align="center"><a href="http://3e.org/"><img src="https://avatars0.githubusercontent.com/u/41439?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Daniel M. Drucker</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=dmd" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/jystervinou"><img src="https://avatars3.githubusercontent.com/u/132356?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Jean-Yves Stervinou</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=jystervinou" title="Code">💻</a></td>
-    <td align="center"><a href="https://dethi.me/"><img src="https://avatars2.githubusercontent.com/u/1011520?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Thibault Deutsch</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=dethi" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/britiscurious"><img src="https://avatars1.githubusercontent.com/u/25646439?v=4?s=75" width="75px;" alt=""/><br /><sub><b>britiscurious</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=britiscurious" title="Documentation">📖</a> <a href="https://github.com/RhetTbull/osxphotos/commits?author=britiscurious" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/mwort"><img src="https://avatars3.githubusercontent.com/u/8170417?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Michel Wortmann</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=mwort" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/PabloKohan"><img src="https://avatars3.githubusercontent.com/u/8790976?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Pablo 'merKur' Kohan</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=PabloKohan" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/hshore29"><img src="https://avatars2.githubusercontent.com/u/7023497?v=4?s=75" width="75px;" alt=""/><br /><sub><b>hshore29</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=hshore29" title="Code">💻</a></td>
+    <td align="center"><a href="http://3e.org/"><img src="https://avatars0.githubusercontent.com/u/41439?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Daniel M. Drucker</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=dmd" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/jystervinou"><img src="https://avatars3.githubusercontent.com/u/132356?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Jean-Yves Stervinou</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=jystervinou" title="Code">💻</a></td>
+    <td align="center"><a href="https://dethi.me/"><img src="https://avatars2.githubusercontent.com/u/1011520?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Thibault Deutsch</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=dethi" title="Code">💻</a></td>
   </tr>
   <tr>
-    <td align="center"><a href="https://github.com/grundsch"><img src="https://avatars0.githubusercontent.com/u/3874928?v=4?s=100" width="100px;" alt=""/><br /><sub><b>grundsch</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=grundsch" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/agprimatic"><img src="https://avatars1.githubusercontent.com/u/4685054?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Ag Primatic</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=agprimatic" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/hhoeck"><img src="https://avatars1.githubusercontent.com/u/6313998?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Horst Höck</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=hhoeck" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/jstrine"><img src="https://avatars1.githubusercontent.com/u/33943447?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Jonathan Strine</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=jstrine" title="Code">💻</a></td>
-    <td align="center"><a href="https://github.com/finestream"><img src="https://avatars1.githubusercontent.com/u/16638513?v=4?s=100" width="100px;" alt=""/><br /><sub><b>finestream</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=finestream" title="Documentation">📖</a></td>
+    <td align="center"><a href="https://github.com/grundsch"><img src="https://avatars0.githubusercontent.com/u/3874928?v=4?s=75" width="75px;" alt=""/><br /><sub><b>grundsch</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=grundsch" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/agprimatic"><img src="https://avatars1.githubusercontent.com/u/4685054?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Ag Primatic</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=agprimatic" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/hhoeck"><img src="https://avatars1.githubusercontent.com/u/6313998?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Horst Höck</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=hhoeck" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/jstrine"><img src="https://avatars1.githubusercontent.com/u/33943447?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Jonathan Strine</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=jstrine" title="Code">💻</a></td>
+    <td align="center"><a href="https://github.com/finestream"><img src="https://avatars1.githubusercontent.com/u/16638513?v=4?s=75" width="75px;" alt=""/><br /><sub><b>finestream</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=finestream" title="Documentation">📖</a></td>
+    <td align="center"><a href="https://github.com/synox"><img src="https://avatars2.githubusercontent.com/u/2250964?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Aravindo Wingeier</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/commits?author=synox" title="Documentation">📖</a></td>
   </tr>
 </table>
 
@@ -2596,6 +2617,7 @@ For additional details about how osxphotos is implemented or if you would like t
 - [pathvalidate](https://pypi.org/project/pathvalidate/)
 - [wurlitzer](https://pypi.org/project/wurlitzer/)
 - [toml](https://github.com/uiri/toml)
+- [PhotoScript](https://github.com/RhetTbull/PhotoScript)
 
 
 ## Acknowledgements

osxphotos/__main__.py

@@ -1345,8 +1345,10 @@ def query(
 @click.option(
     "--export-as-hardlink",
     is_flag=True,
-    help="Hardlink files instead of copying them.  "
-    "Cannot be used with --exiftool which creates copies of the files with embedded EXIF data.",
+    help="Hardlink files instead of copying them. "
+    "Cannot be used with --exiftool which creates copies of the files with embedded EXIF data. "
+    "Note: on APFS volumes, files are cloned when exporting giving many of the same "
+    "advantages as hardlinks without having to use --export-as-hardlink.",
 )
 @click.option(
     "--touch-file",
@@ -2319,6 +2321,8 @@ def export(
                 # but was missing on --update doesn't get deleted
                 # (better to have old version than none)
                 + results.missing
+                # include files that have error in case they exist from previous export
+                + [r[0] for r in results.error]
                 + [str(pathlib.Path(export_db_path).resolve())]
             )
             click.echo(f"Cleaning up {dest}")
@@ -2995,7 +2999,19 @@ def export_photo(
             if export_original:
                 if missing_original:
                     space = " " if not verbose else ""
-                    verbose_(f"{space}Skipping missing photo {photo.original_filename}")
+                    verbose_(
+                        f"{space}Skipping missing photo {photo.original_filename} ({photo.uuid})"
+                    )
+                    results.missing.append(
+                        str(pathlib.Path(dest_path) / original_filename)
+                    )
+                elif photo.intrash and (not photo.path or use_photos_export):
+                    # skip deleted files if they're missing or using use_photos_export
+                    # as AppleScript/PhotoKit cannot export deleted photos
+                    space = " " if not verbose else ""
+                    verbose_(
+                        f"{space}Skipping missing deleted photo {photo.original_filename} ({photo.uuid})"
+                    )
                     results.missing.append(
                         str(pathlib.Path(dest_path) / original_filename)
                     )
@@ -3044,6 +3060,14 @@ def export_photo(
                                 ),
                                 err=True,
                             )
+                        for error_ in export_results.error:
+                            click.echo(
+                                click.style(
+                                    f"Error exporting photo, file {error_[0]}: {error_[1]}",
+                                    fg=CLI_COLOR_ERROR,
+                                ),
+                                err=True,
+                            )
 
                     except Exception as e:
                         click.echo(
@@ -3054,7 +3078,7 @@ def export_photo(
                             err=True,
                         )
                         results.error.append(
-                            str(pathlib.Path(dest) / original_filename)
+                            (str(pathlib.Path(dest) / original_filename), e)
                         )
             else:
                 verbose_(f"Skipping original version of {photo.original_filename}")
@@ -3062,9 +3086,6 @@ def export_photo(
             # if export-edited, also export the edited version
             # verify the photo has adjustments and valid path to avoid raising an exception
             if export_edited and photo.hasadjustments:
-                # if download_missing and the photo is missing or path doesn't exist,
-                # try to download with Photos
-
                 edited_filename = pathlib.Path(filename)
                 # check for correct edited suffix
                 if photo.path_edited is not None:
@@ -3107,10 +3128,21 @@ def export_photo(
                 )
                 if missing_edited:
                     space = " " if not verbose else ""
-                    verbose_(f"{space}Skipping missing edited photo for {filename}")
+                    verbose_(f"{space}Skipping missing edited photo for {edited_filename}")
                     results.missing.append(
                         str(pathlib.Path(dest_path) / edited_filename)
                     )
+                elif photo.intrash and (not photo.path_edited or use_photos_export):
+                    # skip deleted files if they're missing or using use_photos_export
+                    # as AppleScript/PhotoKit cannot export deleted photos
+                    space = " " if not verbose else ""
+                    verbose_(
+                        f"{space}Skipping missing deleted photo {photo.original_filename} ({photo.uuid})"
+                    )
+                    results.missing.append(
+                        str(pathlib.Path(dest_path) / edited_filename )
+                    )
+
                 else:
                     try:
                         export_results_edited = photo.export2(
@@ -3163,7 +3195,9 @@ def export_photo(
                             ),
                             err=True,
                         )
-                        results.error.append(str(pathlib.Path(dest) / edited_filename))
+                        results.error.append(
+                            (str(pathlib.Path(dest) / edited_filename), e)
+                        )
 
             if verbose:
                 if update:
@@ -3367,7 +3401,7 @@ def write_export_report(report_file, results):
             "sidecar_json": 0,
             "sidecar_exiftool": 0,
             "missing": 0,
-            "error": 0,
+            "error": "",
             "exiftool_warning": "",
             "exiftool_error": "",
             "extended_attributes_written": 0,
@@ -3425,7 +3459,7 @@ def write_export_report(report_file, results):
         all_results[result]["missing"] = 1
 
     for result in results.error:
-        all_results[result]["error"] = 1
+        all_results[result[0]]["error"] = result[1]
 
     for result in results.exiftool_warning:
         all_results[result[0]]["exiftool_warning"] = result[1]

osxphotos/_version.py

@@ -1,5 +1,5 @@
 """ version info """
 
-__version__ = "0.39.5"
+__version__ = "0.39.10"
 
 

osxphotos/exiftool.py

@@ -9,11 +9,11 @@
 import json
 import logging
 import os
+import re
 import shutil
 import subprocess
 from functools import lru_cache  # pylint: disable=syntax-error
 
-
 # exiftool -stay_open commands outputs this EOF marker after command is run
 EXIFTOOL_STAYOPEN_EOF = "{ready}"
 EXIFTOOL_STAYOPEN_EOF_LEN = len(EXIFTOOL_STAYOPEN_EOF)
@@ -300,17 +300,28 @@ class ExifTool:
         ver, _, _ = self.run_commands("-ver", no_file=True)
         return ver.decode("utf-8")
 
-    def asdict(self):
+    def asdict(self, tag_groups=True):
         """return dictionary of all EXIF tags and values from exiftool
         returns empty dict if no tags
+
+        Args:
+            tag_groups: if True (default), dict keys have tag groups, e.g. "IPTC:Keywords"; if False, drops groups from keys, e.g. "Keywords"
         """
         json_str, _, _ = self.run_commands("-json")
-        if json_str:
-            exifdict = json.loads(json_str)
-            return exifdict[0]
-        else:
+        if not json_str:
             return dict()
 
+        exifdict = json.loads(json_str)
+        exifdict = exifdict[0]
+        if not tag_groups:
+            # strip tag groups
+            exif_new = {}
+            for k, v in exifdict.items():
+                k = re.sub(r".*:", "", k)
+                exif_new[k] = v
+            exifdict = exif_new
+        return exifdict
+
     def json(self):
         """ returns JSON string containing all EXIF tags and values from exiftool """
         json, _, _ = self.run_commands("-json")

osxphotos/imageconverter.py

@@ -4,7 +4,6 @@
 
 # reference: https://stackoverflow.com/questions/59330149/coreimage-ciimage-write-jpg-is-shifting-colors-macos/59334308#59334308
 
-import logging
 import pathlib
 
 import Metal
@@ -16,6 +15,11 @@ from Foundation import NSDictionary
 from wurlitzer import pipes
 
 
+class ImageConversionError(Exception):
+    """Base class for exceptions in this module. """
+
+    pass
+
 class ImageConverter:
     """ Convert images to jpeg.  This class is a singleton
         which will re-use the Core Image CIContext to avoid
@@ -60,6 +64,7 @@ class ImageConverter:
         Raises:
             ValueError if compression quality not in range 0.0 to 1.0
             FileNotFoundError if input_path doesn't exist
+            ImageConversionError if error during conversion
         """
 
         # accept input_path or output_path as pathlib.Path
@@ -89,8 +94,7 @@ class ImageConverter:
             input_image = Quartz.CIImage.imageWithContentsOfURL_(input_url)
 
         if input_image is None:
-            logging.debug(f"Could not create CIImage for {input_path}")
-            return False
+            raise ImageConversionError(f"Could not create CIImage for {input_path}")
 
         output_colorspace = input_image.colorSpace() or Quartz.CGColorSpaceCreateWithName(
             Quartz.CoreGraphics.kCGColorSpaceSRGB
@@ -105,8 +109,7 @@ class ImageConverter:
         if not error:
             return True
         else:
-            logging.debug(
+            raise ImageConversionError(
                 "Error converting file {input_path} to jpeg at {output_path}: {error}"
             )
-            return False
 

osxphotos/photoinfo/_photoinfo_export.py

@@ -52,6 +52,12 @@ from ..photokit import (
 from ..utils import dd_to_dms_str, findfiles, noop
 
 
+class ExportError(Exception):
+    """ error during export """
+
+    pass
+
+
 class ExportResults:
     """ holds export results for export2 """
 
@@ -114,10 +120,10 @@ class ExportResults:
             + self.sidecar_xmp_written
             + self.sidecar_xmp_skipped
             + self.missing
-            + self.error
         )
         files += [x[0] for x in self.exiftool_warning]
         files += [x[0] for x in self.exiftool_error]
+        files += [x[0] for x in self.error]
 
         files = list(set(files))
         return files
@@ -188,56 +194,33 @@ def _export_photo_uuid_applescript(
 ):
     """Export photo to dest path using applescript to control Photos
     If photo is a live photo, exports both the photo and associated .mov file
-    uuid: UUID of photo to export
-    dest: destination path to export to
-    filestem: (string) if provided, exported filename will be named stem.ext
-              where ext is extension of the file exported by photos (e.g. .jpeg, .mov, etc)
-              If not provided, file will be named with whatever name Photos uses
-              If filestem.ext exists, it wil be overwritten
-    original: (boolean) if True, export original image; default = True
-    edited: (boolean) if True, export edited photo; default = False
-            If photo not edited and edited=True, will still export the original image
-            caller must verify image has been edited
-    *Note*: must be called with either edited or original but not both,
-            will raise error if called with both edited and original = True
-    live_photo: (boolean) if True, export associated .mov live photo; default = False
-    timeout: timeout value in seconds; export will fail if applescript run time exceeds timeout
-    burst: (boolean) set to True if file is a burst image to avoid Photos export error
-    dry_run: (boolean) set to True to run in "dry run" mode which will download file but not actually copy to destination
+
+    Args:
+        uuid: UUID of photo to export
+        dest: destination path to export to
+        filestem: (string) if provided, exported filename will be named stem.ext
+                where ext is extension of the file exported by photos (e.g. .jpeg, .mov, etc)
+                If not provided, file will be named with whatever name Photos uses
+                If filestem.ext exists, it wil be overwritten
+        original: (boolean) if True, export original image; default = True
+        edited: (boolean) if True, export edited photo; default = False
+                If photo not edited and edited=True, will still export the original image
+                caller must verify image has been edited
+        *Note*: must be called with either edited or original but not both,
+                will raise error if called with both edited and original = True
+        live_photo: (boolean) if True, export associated .mov live photo; default = False
+        timeout: timeout value in seconds; export will fail if applescript run time exceeds timeout
+        burst: (boolean) set to True if file is a burst image to avoid Photos export error
+        dry_run: (boolean) set to True to run in "dry run" mode which will download file but not actually copy to destination
+
     Returns: list of paths to exported file(s) or None if export failed
+
+    Raises: ExportError if error during export
+
     Note: For Live Photos, if edited=True, will export a jpeg but not the movie, even if photo
           has not been edited. This is due to how Photos Applescript interface works.
     """
 
-    # setup the applescript to do the export
-    # export_scpt = AppleScript(
-    #     """
-    # 	on export_by_uuid(theUUID, thePath, original, edited, theTimeOut)
-    # 		tell application "Photos"
-    # 			set thePath to thePath
-    # 			set theItem to media item id theUUID
-    # 			set theFilename to filename of theItem
-    # 			set itemList to {theItem}
-
-    # 			if original then
-    # 				with timeout of theTimeOut seconds
-    # 					export itemList to POSIX file thePath with using originals
-    # 				end timeout
-    # 			end if
-
-    # 			if edited then
-    # 				with timeout of theTimeOut seconds
-    # 					export itemList to POSIX file thePath
-    # 				end timeout
-    # 			end if
-
-    # 			return theFilename
-    # 		end tell
-
-    # 	end export_by_uuid
-    # 	"""
-    # )
-
     dest = pathlib.Path(dest)
     if not dest.is_dir():
         raise ValueError(f"dest {dest} must be a directory")
@@ -253,12 +236,8 @@ def _export_photo_uuid_applescript(
         photo = photoscript.Photo(uuid)
         filename = photo.filename
         exported_files = photo.export(tmpdir.name, original=original, timeout=timeout)
-        # filename = export_scpt.call(
-        #     "export_by_uuid", uuid, tmpdir.name, original, edited, timeout
-        # )
     except Exception as e:
-        logging.warning(f"Error exporting uuid {uuid}: {e}")
-        return None
+        raise ExportError(e)
 
     if exported_files and filename:
         # need to find actual filename as sometimes Photos renames JPG to jpeg on export
@@ -527,6 +506,7 @@ def export2(
         "sidecar_xmp_skipped",
         "missing",
         "error",
+        "error_str",
         "exiftool_warning",
         "exiftool_error",
 
@@ -553,24 +533,6 @@ def export2(
     # e.g. name will be filename_edited.jpg
     edited_identifier = "_edited"
 
-    # list of all files exported during this call to export
-    exported_files = []
-
-    # list of new files during update
-    update_new_files = []
-
-    # list of files that were updated
-    update_updated_files = []
-
-    # list of all files skipped because they do not need to be updated (for use with update=True)
-    update_skipped_files = []
-
-    # list of all files with utime touched (touch_file = True)
-    touched_files = []
-
-    # list of all files convereted to jpeg
-    converted_to_jpeg_files = []
-
     # check edited and raise exception trying to export edited version of
     # photo that hasn't been edited
     if edited and not self.hasadjustments:
@@ -645,6 +607,7 @@ def export2(
             f"destination exists ({dest}); overwrite={overwrite}, increment={increment}"
         )
 
+    all_results = ExportResults()
     if not use_photos_export:
         # find the source file on disk and export
         # get path to source file and verify it's not None and is valid file
@@ -738,12 +701,7 @@ def export2(
             jpeg_quality=jpeg_quality,
             ignore_signature=ignore_signature,
         )
-        exported_files = results.exported
-        update_new_files = results.new
-        update_updated_files = results.updated
-        update_skipped_files = results.skipped
-        touched_files = results.touched
-        converted_to_jpeg_files = results.converted_to_jpeg
+        all_results += results
 
         # copy live photo associated .mov if requested
         if live_photo and self.live_photo:
@@ -764,12 +722,7 @@ def export2(
                     fileutil=fileutil,
                     ignore_signature=ignore_signature,
                 )
-                exported_files.extend(results.exported)
-                update_new_files.extend(results.new)
-                update_updated_files.extend(results.updated)
-                update_skipped_files.extend(results.skipped)
-                touched_files.extend(results.touched)
-                converted_to_jpeg_files.extend(results.converted_to_jpeg)
+                all_results += results
 
         # copy associated RAW image if requested
         if raw_photo and self.has_raw:
@@ -791,15 +744,9 @@ def export2(
                     jpeg_quality=jpeg_quality,
                     ignore_signature=ignore_signature,
                 )
-                exported_files.extend(results.exported)
-                update_new_files.extend(results.new)
-                update_updated_files.extend(results.updated)
-                update_skipped_files.extend(results.skipped)
-                touched_files.extend(results.touched)
-                converted_to_jpeg_files.extend(results.converted_to_jpeg)
+                all_results += results
     else:
         # use_photo_export
-        exported = []
         # export live_photo .mov file?
         live_photo = True if live_photo and self.live_photo else False
         if edited or self.shared:
@@ -819,7 +766,7 @@ def export2(
                 photo = None
                 try:
                     photo = photolib.fetch_uuid(self.uuid)
-                except PhotoKitFetchFailed:
+                except PhotoKitFetchFailed as e:
                     # if failed to find UUID, might be a burst photo
                     if self.burst and self._info["burstUUID"]:
                         bursts = photolib.fetch_burst_uuid(
@@ -828,24 +775,32 @@ def export2(
                         # PhotoKit UUIDs may contain "/L0/001" so only look at beginning
                         photo = [p for p in bursts if p.uuid.startswith(self.uuid)]
                         photo = photo[0] if photo else None
+                    if not photo:
+                        all_results.error.append((str(dest), f"PhotoKitFetchFailed exception exporting photo {self.uuid}: {e}"))
                 if photo:
-                    exported = photo.export(
-                        dest.parent, dest.name, version=PHOTOS_VERSION_CURRENT
-                    )
-                else:
-                    exported = []
+                    try:
+                        exported = photo.export(
+                            dest.parent, dest.name, version=PHOTOS_VERSION_CURRENT
+                        )
+                        all_results.exported.extend(exported)
+                    except Exception as e:
+                        all_results.error.append((str(dest), e))
             else:
-                exported = _export_photo_uuid_applescript(
-                    self.uuid,
-                    dest.parent,
-                    filestem=filestem,
-                    original=False,
-                    edited=True,
-                    live_photo=live_photo,
-                    timeout=timeout,
-                    burst=self.burst,
-                    dry_run=dry_run,
-                )
+                try:
+                    exported = _export_photo_uuid_applescript(
+                        self.uuid,
+                        dest.parent,
+                        filestem=filestem,
+                        original=False,
+                        edited=True,
+                        live_photo=live_photo,
+                        timeout=timeout,
+                        burst=self.burst,
+                        dry_run=dry_run,
+                    )
+                    all_results.exported.extend(exported)
+                except ExportError as e:
+                    all_results.error.append((str(dest), e))
         else:
             # export original version and not edited
             filestem = dest.stem
@@ -864,13 +819,16 @@ def export2(
                         photo = [p for p in bursts if p.uuid.startswith(self.uuid)]
                         photo = photo[0] if photo else None
                 if photo:
-                    exported = photo.export(
+                    try:
+                        exported = photo.export(
                         dest.parent, dest.name, version=PHOTOS_VERSION_ORIGINAL
-                    )
-                else:
-                    exported = []
+                        )
+                        all_results.exported.extend(exported)
+                    except Exception as e:
+                        all_results.error.append((str(dest), e))
             else:
-                exported = _export_photo_uuid_applescript(
+                try:
+                    exported = _export_photo_uuid_applescript(
                     self.uuid,
                     dest.parent,
                     filestem=filestem,
@@ -880,21 +838,21 @@ def export2(
                     timeout=timeout,
                     burst=self.burst,
                     dry_run=dry_run,
-                )
-        if exported:
+                    )
+                    all_results.exported.extend(exported)
+                except ExportError as e:
+                    all_results.error.append((str(dest), e))
+        if all_results.exported:
             if touch_file:
-                for exported_file in exported:
-                    touched_files.append(exported_file)
+                for exported_file in all_results.exported:
+                    all_results.touched.append(exported_file)
                     ts = int(self.date.timestamp())
                     fileutil.utime(exported_file, (ts, ts))
-            exported_files.extend(exported)
             if update:
-                update_new_files.extend(exported)
+                all_results.new.extend(all_results.exported)
 
-        else:
-            logging.warning(
-                f"Error exporting photo {self.uuid} to {dest} with use_photos_export"
-            )
+        # else:
+            # all_results.error.append((str(dest), f"Error exporting photo {self.uuid} to {dest} with use_photos_export"))
 
     # export metadata
     sidecars = []
@@ -1006,14 +964,10 @@ def export2(
 
     # if exiftool, write the metadata
     if update:
-        exif_files = update_new_files + update_updated_files + update_skipped_files
+        exif_files = all_results.new + all_results.updated + all_results.skipped
     else:
-        exif_files = exported_files
+        exif_files = all_results.exported
 
-    exif_files_updated = []
-    exiftool_warning = []
-    exiftool_error = []
-    errors = []
     # TODO: remove duplicative code from below
     if exiftool and update and exif_files:
         for exported_file in exif_files:
@@ -1052,10 +1006,10 @@ def export2(
                         merge_exif_persons=merge_exif_persons,
                     )
                     if warning_:
-                        exiftool_warning.append((exported_file, warning_))
+                        all_results.exiftool_warning.append((exported_file, warning_))
                     if error_:
-                        exiftool_error.append((exported_file, error_))
-                        errors.append(exported_file)
+                        all_results.exiftool_error.append((exported_file, error_))
+                        all_results.error.append((exported_file, error_))
 
                 export_db.set_exifdata_for_file(
                     exported_file,
@@ -1072,7 +1026,7 @@ def export2(
                 export_db.set_stat_exif_for_file(
                     exported_file, fileutil.file_sig(exported_file)
                 )
-                exif_files_updated.append(exported_file)
+                all_results.exif_updated.append(exported_file)
             else:
                 verbose(f"Skipped up to date exiftool metadata for {exported_file}")
     elif exiftool and exif_files:
@@ -1091,10 +1045,10 @@ def export2(
                     merge_exif_persons=merge_exif_persons,
                 )
                 if warning_:
-                    exiftool_warning.append((exported_file, warning_))
+                    all_results.exiftool_warning.append((exported_file, warning_))
                 if error_:
-                    exiftool_error.append((exported_file, error_))
-                    errors.append(exported_file)
+                    all_results.exiftool_error.append((exported_file, error_))
+                    all_results.error.append((exported_file, error_))
 
             export_db.set_exifdata_for_file(
                 exported_file,
@@ -1111,36 +1065,25 @@ def export2(
             export_db.set_stat_exif_for_file(
                 exported_file, fileutil.file_sig(exported_file)
             )
-            exif_files_updated.append(exported_file)
+            all_results.exif_updated.append(exported_file)
 
     if touch_file:
-        for exif_file in exif_files_updated:
+        for exif_file in all_results.exif_updated:
             verbose(f"Updating file modification time for {exif_file}")
-            touched_files.append(exif_file)
+            all_results.touched.append(exif_file)
             ts = int(self.date.timestamp())
             fileutil.utime(exif_file, (ts, ts))
 
-    touched_files = list(set(touched_files))
+    all_results.touched = list(set(all_results.touched))
 
-    results = ExportResults(
-        exported=exported_files,
-        new=update_new_files,
-        updated=update_updated_files,
-        skipped=update_skipped_files,
-        exif_updated=exif_files_updated,
-        touched=touched_files,
-        converted_to_jpeg=converted_to_jpeg_files,
-        sidecar_json_written=sidecar_json_files_written,
-        sidecar_json_skipped=sidecar_json_files_skipped,
-        sidecar_exiftool_written=sidecar_exiftool_files_written,
-        sidecar_exiftool_skipped=sidecar_exiftool_files_skipped,
-        sidecar_xmp_written=sidecar_xmp_files_written,
-        sidecar_xmp_skipped=sidecar_xmp_files_skipped,
-        error=errors,
-        exiftool_error=exiftool_error,
-        exiftool_warning=exiftool_warning,
-    )
-    return results
+    all_results.sidecar_json_written = sidecar_json_files_written
+    all_results.sidecar_json_skipped = sidecar_json_files_skipped
+    all_results.sidecar_exiftool_written = sidecar_exiftool_files_written
+    all_results.sidecar_exiftool_skipped = sidecar_exiftool_files_skipped
+    all_results.sidecar_xmp_written = sidecar_xmp_files_written
+    all_results.sidecar_xmp_skipped = sidecar_xmp_files_skipped
+
+    return all_results
 
 
 def _export_photo(

setup.py

@@ -51,7 +51,7 @@ with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
 setup(
     name="osxphotos",
     version=about["__version__"],
-    description="Manipulate (read-only) Apple's Photos app library on Mac OS X",
+    description="Export photos from Apple's macOS Photos app and query the Photos library database to access metadata about images.",
     long_description=about["long_description"],
     long_description_content_type="text/markdown",
     author="Rhet Turnbull",

tests/Test-10.16.0.1.photoslibrary/database/DataModelVersion.plist

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>LibrarySchemaVersion</key>
+	<integer>5001</integer>
+	<key>MetaSchemaVersion</key>
+	<integer>3</integer>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/database/Photos.sqlite.lock

@@ -0,0 +1,16 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>hostname</key>
+	<string>Rhets-MacBook-Pro.local</string>
+	<key>hostuuid</key>
+	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
+	<key>pid</key>
+	<integer>400</integer>
+	<key>processname</key>
+	<string>photolibraryd</string>
+	<key>uid</key>
+	<integer>501</integer>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/database/search/searchMetadata.plist

@@ -0,0 +1,188 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>BlacklistedMeaningsByMeaning</key>
+	<dict/>
+	<key>MePersonUUID</key>
+	<string>39488755-78C0-40B2-B378-EDA280E1823C</string>
+	<key>SceneWhitelist</key>
+	<array>
+		<string>Graduation</string>
+		<string>Aquarium</string>
+		<string>Food</string>
+		<string>Ice Skating</string>
+		<string>Mountain</string>
+		<string>Cliff</string>
+		<string>Basketball</string>
+		<string>Tennis</string>
+		<string>Jewelry</string>
+		<string>Cheese</string>
+		<string>Softball</string>
+		<string>Football</string>
+		<string>Circus</string>
+		<string>Jet Ski</string>
+		<string>Playground</string>
+		<string>Carousel</string>
+		<string>Paint Ball</string>
+		<string>Windsurfing</string>
+		<string>Sailboat</string>
+		<string>Sunbathing</string>
+		<string>Dam</string>
+		<string>Fireplace</string>
+		<string>Flower</string>
+		<string>Scuba</string>
+		<string>Hiking</string>
+		<string>Cetacean</string>
+		<string>Pier</string>
+		<string>Bowling</string>
+		<string>Snowboarding</string>
+		<string>Zoo</string>
+		<string>Snowmobile</string>
+		<string>Theater</string>
+		<string>Boat</string>
+		<string>Casino</string>
+		<string>Car</string>
+		<string>Diving</string>
+		<string>Cycling</string>
+		<string>Musical Instrument</string>
+		<string>Board Game</string>
+		<string>Castle</string>
+		<string>Sunset Sunrise</string>
+		<string>Martial Arts</string>
+		<string>Motocross</string>
+		<string>Submarine</string>
+		<string>Cat</string>
+		<string>Snow</string>
+		<string>Kiteboarding</string>
+		<string>Squash</string>
+		<string>Geyser</string>
+		<string>Music</string>
+		<string>Archery</string>
+		<string>Desert</string>
+		<string>Blackjack</string>
+		<string>Fireworks</string>
+		<string>Sportscar</string>
+		<string>Feline</string>
+		<string>Soccer</string>
+		<string>Museum</string>
+		<string>Baby</string>
+		<string>Fencing</string>
+		<string>Railroad</string>
+		<string>Nascar</string>
+		<string>Sky Surfing</string>
+		<string>Bird</string>
+		<string>Games</string>
+		<string>Baseball</string>
+		<string>Dressage</string>
+		<string>Snorkeling</string>
+		<string>Pyramid</string>
+		<string>Kite</string>
+		<string>Rowboat</string>
+		<string>Golf</string>
+		<string>Watersports</string>
+		<string>Lightning</string>
+		<string>Canyon</string>
+		<string>Auditorium</string>
+		<string>Night Sky</string>
+		<string>Karaoke</string>
+		<string>Skiing</string>
+		<string>Parade</string>
+		<string>Forest</string>
+		<string>Hot Air Balloon</string>
+		<string>Dragon Parade</string>
+		<string>Easter Egg</string>
+		<string>Monument</string>
+		<string>Jungle</string>
+		<string>Thanksgiving</string>
+		<string>Jockey Horse</string>
+		<string>Stadium</string>
+		<string>Airplane</string>
+		<string>Ballet</string>
+		<string>Yoga</string>
+		<string>Coral Reef</string>
+		<string>Skating</string>
+		<string>Wrestling</string>
+		<string>Bicycle</string>
+		<string>Tattoo</string>
+		<string>Amusement Park</string>
+		<string>Canoe</string>
+		<string>Cheerleading</string>
+		<string>Ping Pong</string>
+		<string>Fishing</string>
+		<string>Magic</string>
+		<string>Reptile</string>
+		<string>Winter Sport</string>
+		<string>Waterfall</string>
+		<string>Train</string>
+		<string>Bonsai</string>
+		<string>Surfing</string>
+		<string>Dog</string>
+		<string>Cake</string>
+		<string>Sledding</string>
+		<string>Sandcastle</string>
+		<string>Glacier</string>
+		<string>Lighthouse</string>
+		<string>Equestrian</string>
+		<string>Rafting</string>
+		<string>Shore</string>
+		<string>Hockey</string>
+		<string>Santa Claus</string>
+		<string>Formula One Car</string>
+		<string>Sport</string>
+		<string>Vehicle</string>
+		<string>Boxing</string>
+		<string>Rollerskating</string>
+		<string>Underwater</string>
+		<string>Orchestra</string>
+		<string>Carnival</string>
+		<string>Rocket</string>
+		<string>Skateboarding</string>
+		<string>Helicopter</string>
+		<string>Performance</string>
+		<string>Oktoberfest</string>
+		<string>Water Polo</string>
+		<string>Skate Park</string>
+		<string>Animal</string>
+		<string>Nightclub</string>
+		<string>String Instrument</string>
+		<string>Dinosaur</string>
+		<string>Gymnastics</string>
+		<string>Cricket</string>
+		<string>Volcano</string>
+		<string>Lake</string>
+		<string>Aurora</string>
+		<string>Dancing</string>
+		<string>Concert</string>
+		<string>Rock Climbing</string>
+		<string>Hang Glider</string>
+		<string>Rodeo</string>
+		<string>Fish</string>
+		<string>Art</string>
+		<string>Motorcycle</string>
+		<string>Volleyball</string>
+		<string>Wake Boarding</string>
+		<string>Badminton</string>
+		<string>Motor Sport</string>
+		<string>Sumo</string>
+		<string>Parasailing</string>
+		<string>Skydiving</string>
+		<string>Kickboxing</string>
+		<string>Pinata</string>
+		<string>Foosball</string>
+		<string>Go Kart</string>
+		<string>Poker</string>
+		<string>Kayak</string>
+		<string>Swimming</string>
+		<string>Atv</string>
+		<string>Beach</string>
+		<string>Dartboard</string>
+		<string>Athletics</string>
+		<string>Camping</string>
+		<string>Tornado</string>
+		<string>Billiards</string>
+		<string>Rugby</string>
+		<string>Airshow</string>
+	</array>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/database/search/searchProgress.plist

@@ -0,0 +1,26 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>insertAlbum</key>
+	<array/>
+	<key>insertAsset</key>
+	<array/>
+	<key>insertHighlight</key>
+	<array/>
+	<key>insertMemory</key>
+	<array/>
+	<key>insertMoment</key>
+	<array/>
+	<key>removeAlbum</key>
+	<array/>
+	<key>removeAsset</key>
+	<array/>
+	<key>removeHighlight</key>
+	<array/>
+	<key>removeMemory</key>
+	<array/>
+	<key>removeMoment</key>
+	<array/>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/database/search/searchSystemInfo.plist

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>embeddingVersion</key>
+	<string>1</string>
+	<key>localeIdentifier</key>
+	<string>en_US</string>
+	<key>sceneTaxonomySHA</key>
+	<string>87914a047c69fbe8013fad2c70fa70c6c03b08b56190fe4054c880e6b9f57cc3</string>
+	<key>searchIndexVersion</key>
+	<string>10</string>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.Photos.Migration/appPrivateData.plist

@@ -0,0 +1,26 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>MigrationService</key>
+	<dict>
+		<key>State</key>
+		<integer>4</integer>
+	</dict>
+	<key>MigrationService.LastCompletedTask</key>
+	<integer>12</integer>
+	<key>MigrationService.ValidationCounts</key>
+	<dict>
+		<key>MigrationDetectedFaceprint</key>
+		<integer>6</integer>
+		<key>MigrationManagedAsset</key>
+		<integer>0</integer>
+		<key>MigrationSceneClassification</key>
+		<integer>44</integer>
+		<key>MigrationUnmanagedAdjustment</key>
+		<integer>0</integer>
+		<key>RDVersion.cloudLocalState.CPLIsNotPushed</key>
+		<integer>7</integer>
+	</dict>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -0,0 +1,58 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>CollapsedSidebarSectionIdentifiers</key>
+	<array/>
+	<key>ExpandedSidebarItemIdentifiers</key>
+	<array>
+		<string>92D68107-B6C7-453B-96D2-97B0F26D5B8B/L0/020</string>
+		<string>88A5F8B8-5B9A-43C7-BB85-3952B81580EB/L0/020</string>
+		<string>29EF7A97-7E76-4D5F-A5E0-CC0A93E8524C/L0/020</string>
+		<string>2C2AF115-BD1D-4434-A747-D1C8BD8E2045/L0/020</string>
+		<string>CB051A4C-2CB7-4B90-B59B-08CC4D0C2823/L0/020</string>
+	</array>
+	<key>IPXWorkspaceControllerZoomLevelsKey</key>
+	<dict>
+		<key>kZoomLevelIdentifierPhotosGrid</key>
+		<integer>2</integer>
+	</dict>
+	<key>Photos</key>
+	<dict>
+		<key>CollapsedSidebarSectionIdentifiers</key>
+		<array/>
+		<key>ExpandedSidebarItemIdentifiers</key>
+		<array>
+			<string>TopLevelAlbums</string>
+			<string>TopLevelSlideshows</string>
+		</array>
+		<key>IPXWorkspaceControllerZoomLevelsKey</key>
+		<dict>
+			<key>kZoomLevelIdentifierAlbums</key>
+			<integer>7</integer>
+			<key>kZoomLevelIdentifierVersions</key>
+			<integer>7</integer>
+		</dict>
+		<key>lastAddToDestination</key>
+		<dict>
+			<key>key</key>
+			<integer>1</integer>
+			<key>lastKnownDisplayName</key>
+			<string>September 28, 2018</string>
+			<key>type</key>
+			<string>album</string>
+			<key>uuid</key>
+			<string>DFFKmHt3Tk+AGzZLe2Xq+g</string>
+		</dict>
+		<key>lastKnownItemCounts</key>
+		<dict>
+			<key>other</key>
+			<integer>0</integer>
+			<key>photos</key>
+			<integer>7</integer>
+			<key>videos</key>
+			<integer>0</integer>
+		</dict>
+	</dict>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist

@@ -0,0 +1,26 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>BackgroundHighlightCollection</key>
+	<date>2020-06-24T04:02:13Z</date>
+	<key>BackgroundHighlightEnrichment</key>
+	<date>2020-06-24T04:02:12Z</date>
+	<key>BackgroundJobAssetRevGeocode</key>
+	<date>2020-06-24T04:02:13Z</date>
+	<key>BackgroundJobSearch</key>
+	<date>2020-06-24T04:02:13Z</date>
+	<key>BackgroundPeopleSuggestion</key>
+	<date>2020-06-24T04:02:12Z</date>
+	<key>BackgroundUserBehaviorProcessor</key>
+	<date>2020-06-24T04:02:13Z</date>
+	<key>PhotoAnalysisGraphLastBackgroundGraphConsistencyUpdateJobDateKey</key>
+	<date>2020-05-30T02:16:06Z</date>
+	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
+	<date>2020-05-29T04:31:37Z</date>
+	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
+	<date>2020-06-24T04:02:13Z</date>
+	<key>SiriPortraitDonation</key>
+	<date>2020-06-24T04:02:13Z</date>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/revgeoprovider.plist

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>revgeoprovider</key>
+	<string>7618</string>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PersonPromoter

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>NumberOfFacesProcessedOnLastRun</key>
+	<integer>7</integer>
+	<key>ProcessedInQuiescentState</key>
+	<true/>
+	<key>SuggestedMeIdentifier</key>
+	<string></string>
+	<key>Version</key>
+	<integer>4</integer>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>FaceIDModelLastGenerationKey</key>
+	<date>2020-05-29T03:44:04Z</date>
+	<key>LastContactClassificationKey</key>
+	<date>2020-05-29T04:31:40Z</date>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/clustererState.plist

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>PVClustererBringUpState</key>
+	<integer>50</integer>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/faceWorkerState.plist

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>IncrementalPersonProcessingStage</key>
+	<integer>6</integer>
+	<key>PersonBuilderLastMinimumFaceGroupSizeForCreatingMergeCandidates</key>
+	<integer>15</integer>
+	<key>PersonBuilderMergeCandidatesEnabled</key>
+	<true/>
+</dict>
+</plist>

tests/Test-10.16.0.1.photoslibrary/private/com.apple.photoanalysisd/graph/PhotoAnalysisServicePreferences.plist

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
+	<date>2019-08-07T02:26:15Z</date>
+	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
+	<date>2019-08-17T14:26:34Z</date>
+</dict>
+</plist>