Added try/except for bad datettime values

Catch illegal timestamp value

.github/workflows/pythonpackage.yml

@@ -9,7 +9,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [3.6, 3.7, 3.8]
+        python-version: [3.8]
 
     steps:
     - uses: actions/checkout@v1
@@ -21,8 +21,8 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install -r requirements.txt
-    - name: Lint with flake8
-      run: |
+   # - name: Lint with flake8
+   #   run: |
        # pip install flake8
         # stop the build if there are Python syntax errors or undefined names
        # flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
@@ -31,4 +31,5 @@ jobs:
     - name: Test with pytest
       run: |
         pip install pytest
+        pip install pytest-mock
         python -m pytest tests/

CHANGELOG.md

@@ -4,6 +4,120 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.29.0](https://github.com/RhetTbull/osxphotos/compare/v0.28.19...v0.29.0)
+
+> 23 May 2020
+
+- Made --exiftool and --export-as-hardlink incompatible in CLI to fix #132 [`#132`](https://github.com/RhetTbull/osxphotos/issues/132)
+- Added --update to CLI export; reference issue #100 [`b1171e9`](https://github.com/RhetTbull/osxphotos/commit/b1171e96cc06362555725995bb311317eb163e49)
+- Added as_dict to PlaceInfo [`8c4fe40`](https://github.com/RhetTbull/osxphotos/commit/8c4fe40aa6850f166e526cffaa088550884399af)
+- Updated README.md [`11d368a`](https://github.com/RhetTbull/osxphotos/commit/11d368a69cbe67e909e64b020f0334fc09dd3ac4)
+- Updated CHANGELOG.md [`cafa483`](https://github.com/RhetTbull/osxphotos/commit/cafa483cfc228c651a03d3361d6d48a35deab1e8)
+- version bump [`c06c230`](https://github.com/RhetTbull/osxphotos/commit/c06c230a469754691d11fff1034fb02daeeba649)
+
+#### [v0.28.19](https://github.com/RhetTbull/osxphotos/compare/v0.28.18...v0.28.19)
+
+> 15 May 2020
+
+- Added label and label_normalized to template system, closes #130 [`#130`](https://github.com/RhetTbull/osxphotos/issues/130)
+- Revert "test library updates" [`48e9c32`](https://github.com/RhetTbull/osxphotos/commit/48e9c32add549e66c3ef8c65f8821f5033b55b11)
+- test library updates [`d125854`](https://github.com/RhetTbull/osxphotos/commit/d125854f2a04e37747af3e0796370a565c1c9bd0)
+- Updated CHANGELOG.md [`e228cfa`](https://github.com/RhetTbull/osxphotos/commit/e228cfab746055c8d6df428aebe0ed001fb6d4d0)
+- version bump [`bd9d5a2`](https://github.com/RhetTbull/osxphotos/commit/bd9d5a26f3bfcbb33896a139fa86cdab46768103)
+- Update README.md [`85760dc`](https://github.com/RhetTbull/osxphotos/commit/85760dc4fe2274d826ed80494fd4e66866398609)
+
+#### [v0.28.18](https://github.com/RhetTbull/osxphotos/compare/v0.28.17...v0.28.18)
+
+> 14 May 2020
+
+- Implemented PhotoInfo.exiftool [`a80dee4`](https://github.com/RhetTbull/osxphotos/commit/a80dee401c7eb959f6ad6d93a3272657ed28f521)
+- Updated CHANGELOG.md [`e67fce2`](https://github.com/RhetTbull/osxphotos/commit/e67fce28714cf4065b64202bb3b149ba5bec5be4)
+
+#### [v0.28.17](https://github.com/RhetTbull/osxphotos/compare/v0.28.15...v0.28.17)
+
+> 14 May 2020
+
+- Added ExifInfo (Photos 5 only) [`53304d7`](https://github.com/RhetTbull/osxphotos/commit/53304d702317d007056c1d12064503c3ec4ae6f6)
+- Added as_dict to ExifTool [`d1af14d`](https://github.com/RhetTbull/osxphotos/commit/d1af14dbb4d441a62d352123774e51fa3538db97)
+
+#### [v0.28.15](https://github.com/RhetTbull/osxphotos/compare/v0.28.13...v0.28.15)
+
+> 11 May 2020
+
+- fixed some minor findings... [`#127`](https://github.com/RhetTbull/osxphotos/pull/127)
+- added --export-as-hardlink option [`#126`](https://github.com/RhetTbull/osxphotos/pull/126)
+- Added test for folder_names on 10.15.4, closes #119 [`#119`](https://github.com/RhetTbull/osxphotos/issues/119)
+- Refactored photosdb and photoinfo to add SearchInfo and labels [`98b3f63`](https://github.com/RhetTbull/osxphotos/commit/98b3f63a92aa2105f8fa97af992fc6fe2d78b973)
+- Added additional test for --export-as-hardlink [`57315d4`](https://github.com/RhetTbull/osxphotos/commit/57315d44497fde977956f76f667470208f11aa2d)
+- added CHANGELOG.md [`00e1661`](https://github.com/RhetTbull/osxphotos/commit/00e16611fc86c05fb090d036084db9eb42444071)
+- Updated a couple of tests to use pytest-mock [`397db0d`](https://github.com/RhetTbull/osxphotos/commit/397db0d72fb218669a9ecbff134fa9b392a14661)
+- added test for export using hardlinks, fixed a test that failed if users locale settings were different to en_US [`b0ec6c6`](https://github.com/RhetTbull/osxphotos/commit/b0ec6c6b36d8cfe05723d47b210d9d7c5aabdfe5)
+
+#### [v0.28.13](https://github.com/RhetTbull/osxphotos/compare/v0.28.10...v0.28.13)
+
+> 2 May 2020
+
+- added --keyword-template [`65674f5`](https://github.com/RhetTbull/osxphotos/commit/65674f57bc174c078e6c47f12ba3aaba87bfa3a4)
+- Fixed bug related to issue #119 [`7af1ccd`](https://github.com/RhetTbull/osxphotos/commit/7af1ccd4ed22ea7f0f86973bfba7f108b6650291)
+- test library updates [`1b6f661`](https://github.com/RhetTbull/osxphotos/commit/1b6f661e6b59c003d3b8cb35226ffb51469be508)
+
+#### [v0.28.10](https://github.com/RhetTbull/osxphotos/compare/v0.28.8...v0.28.10)
+
+> 29 April 2020
+
+- Bug fix for albums in Photos <= 4 to address issue #116 [`a57da23`](https://github.com/RhetTbull/osxphotos/commit/a57da2346b282d731ed41db600bfc5cbeb1a0992)
+- version bump for pypi [`3fe03cd`](https://github.com/RhetTbull/osxphotos/commit/3fe03cd12752c2a7769007b6d934f1efe9f9c4d2)
+
+#### [v0.28.8](https://github.com/RhetTbull/osxphotos/compare/v0.28.7...v0.28.8)
+
+> 28 April 2020
+
+- Fixed implementation of use_albums_as_keywords and use_persons_as_keywords, closes #115 [`#115`](https://github.com/RhetTbull/osxphotos/issues/115)
+- Updated CHANGELOG.md [`072a8d7`](https://github.com/RhetTbull/osxphotos/commit/072a8d795e5e15fa8ca8d8872aecf4cddd7837f7)
+- Update README.md [`5cc98c3`](https://github.com/RhetTbull/osxphotos/commit/5cc98c338bcc19fd05bf293eb3afe24c07c8b380)
+- Updated README.md [`a800711`](https://github.com/RhetTbull/osxphotos/commit/a80071111f810a1d7d6e2d735839e85499091ea4)
+- Update README.md [`1c9d4f2`](https://github.com/RhetTbull/osxphotos/commit/1c9d4f282beea2ac12273c8d0f9453bad1255c2c)
+
+#### [v0.28.7](https://github.com/RhetTbull/osxphotos/compare/v0.28.6...v0.28.7)
+
+> 28 April 2020
+
+- Added --album-keyword and --person-keyword to CLI, closes #61 [`#61`](https://github.com/RhetTbull/osxphotos/issues/61)
+- Updated test libraries [`54d5d4b`](https://github.com/RhetTbull/osxphotos/commit/54d5d4b7ba99204f58e723231309ab6e306be28c)
+- Updated CHANGELOG.md [`38137a1`](https://github.com/RhetTbull/osxphotos/commit/38137a1351cdb7ab72393ea03828933dac0b76b0)
+- Updated tests/README.md [`56a0006`](https://github.com/RhetTbull/osxphotos/commit/56a000609f2f08d0f8800fec49cada2980c3bb9d)
+
+#### [v0.28.6](https://github.com/RhetTbull/osxphotos/compare/v0.28.5...v0.28.6)
+
+> 26 April 2020
+
+- Fixed locale bug in templates, closes #113 [`#113`](https://github.com/RhetTbull/osxphotos/issues/113)
+- Updated CHANGELOG.md [`81d4e39`](https://github.com/RhetTbull/osxphotos/commit/81d4e392c39f0fe6f967a447c7d0c970bf224032)
+- Updated test to avoid issue with GitHub workflow [`9be0f84`](https://github.com/RhetTbull/osxphotos/commit/9be0f849b73061d053d30274ff3295b79c88f0b6)
+- Update pythonpackage.yml to remove older pythons [`ccb5f25`](https://github.com/RhetTbull/osxphotos/commit/ccb5f252d14e9335ae04a2e338a6d527b80c9a93)
+
+#### [v0.28.5](https://github.com/RhetTbull/osxphotos/compare/0.28.2...v0.28.5)
+
+> 21 April 2020
+
+- added __len__ to PhotosDB, closes #44 [`#44`](https://github.com/RhetTbull/osxphotos/issues/44)
+- Updated use of _PHOTOS_4_VERSION, closes #106 [`#106`](https://github.com/RhetTbull/osxphotos/issues/106)
+- Updated tests and test library with RAW images [`9b9b54e`](https://github.com/RhetTbull/osxphotos/commit/9b9b54e590e43ae49fb3ae41d493a1f8faec4181)
+- Updated setup.py to resolve issue with bpylist2 on python < 3.8 [`8e4b88a`](https://github.com/RhetTbull/osxphotos/commit/8e4b88ad1fc18438f941e045bfc8aeac878914f9)
+- Added cli.py for use with pyinstaller [`cf28cb6`](https://github.com/RhetTbull/osxphotos/commit/cf28cb6452de17f2ef8d80435386e8d5a1aabd34)
+- added raw_is_original handling [`a337e79`](https://github.com/RhetTbull/osxphotos/commit/a337e79e13802b4824c2f088ce9db1c027d6f3c5)
+- Updated CHANGELOG.md [`22f1e8f`](https://github.com/RhetTbull/osxphotos/commit/22f1e8f2a6478e0576f6bff53e348aad8680ae69)
+
+#### [0.28.2](https://github.com/RhetTbull/osxphotos/compare/v0.28.1...0.28.2)
+
+> 18 April 2020
+
+- Added folder support for Photos <= 4, closes #93 [`#93`](https://github.com/RhetTbull/osxphotos/issues/93)
+- cleaned up SQL statements in _process_database4 [`6f28171`](https://github.com/RhetTbull/osxphotos/commit/6f281711e2001a63ffad076d7b9835272d5d09da)
+- Updated CHANGELOG.md [`1fa9583`](https://github.com/RhetTbull/osxphotos/commit/1fa9583ea689d54d2613a064f1ade25bcdfbf043)
+- Fixed suffix check on export to be case insensitive [`4b30b3b`](https://github.com/RhetTbull/osxphotos/commit/4b30b3b4260e2c7409e18825e5b626efe646db16)
+- test library update [`3bac106`](https://github.com/RhetTbull/osxphotos/commit/3bac106eb7a180e9e39643a89087d92bf2a437d0)
+
 #### [v0.28.1](https://github.com/RhetTbull/osxphotos/compare/v0.27.4...v0.28.1)
 
 > 18 April 2020
@@ -273,7 +387,11 @@ Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 - Moved PhotosDB attributes to properties instead of methods [`d95acdf`](https://github.com/RhetTbull/osxphotos/commit/d95acdf9f8764a1720bcba71a6dad29bf668eaf9)
 - changed interface for export, prepped for exiftool_json_sidecar [`1fe8859`](https://github.com/RhetTbull/osxphotos/commit/1fe885962e8a9a420e776bdd3dc640ca143224b2)
 
-#### [v0.15.1](https://github.com/RhetTbull/osxphotos/compare/v0.14.21...v0.15.1)
+#### [v0.15.1](https://github.com/RhetTbull/osxphotos/compare/v0.15.0...v0.15.1)
+
+> 23 May 2020
+
+#### [v0.15.0](https://github.com/RhetTbull/osxphotos/compare/v0.14.21...v0.15.0)
 
 > 14 December 2019
 

README.md

@@ -2,7 +2,7 @@
 
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-
+![Python package](https://github.com/RhetTbull/osxphotos/workflows/Python%20package/badge.svg)
 
 - [OSXPhotos](#osxphotos)
   * [What is osxphotos?](#what-is-osxphotos)
@@ -13,10 +13,11 @@
   * [Package Interface](#package-interface)
     + [PhotosDB](#photosdb)
     + [PhotoInfo](#photoinfo)
+    + [ExifInfo](#exifinfo)
     + [AlbumInfo](#albuminfo)
     + [FolderInfo](#folderinfo)
     + [PlaceInfo](#placeinfo)
-    + [Template Functions](#template-functions)
+    + [Template Substitutions](#template-substitutions)
     + [Utility Functions](#utility-functions)
   * [Examples](#examples)
   * [Related Projects](#related-projects)
@@ -33,17 +34,27 @@ OSXPhotos provides the ability to interact with and query Apple's Photos.app lib
 
 ## Supported operating systems
 
-Only works on MacOS (aka Mac OS X). Tested on MacOS 10.12.6 / Photos 2.0, 10.13.6 / Photos 3.0, MacOS 10.14.5, 10.14.6 / Photos 4.0, MacOS 10.15.1 / Photos 5.0. Requires python >= 3.6
+Only works on MacOS (aka Mac OS X). Tested on MacOS 10.12.6 / Photos 2.0, 10.13.6 / Photos 3.0, MacOS 10.14.5, 10.14.6 / Photos 4.0, MacOS 10.15.1 & 10.15.4 / Photos 5.0. 
 
-This package will read Photos databases for any supported version on any supported OS version.  E.g. you can read a database created with Photos 4.0 on MacOS 10.14 on a machine running MacOS 10.12
+Requires python >= 3.8. You can probably get this to run with Python 3.6 or 3.7 (see notes [below](#Installation-instructions)) but only 3.8+ is officially supported. 
+
+This package will read Photos databases for any supported version on any supported OS version.  E.g. you can read a database created with Photos 4.0 on MacOS 10.14 on a machine running MacOS 10.12.
 
 
 ## Installation instructions
 
-osxmetadata uses setuptools, thus simply run:
+OSXPhotos uses setuptools, thus simply run:
 
 	python3 setup.py install
 
+If you're using python 3.6 or 3.7, you'll need to do this first to get around an issue with bpylist2:
+
+	pip install -r requirements.txt
+
+You can also install directly from [pypi](https://pypi.org/) but you must use python >= 3.8 to avoid an error with bpylist2.  The package currently works fine with python 3.6 or 3.7 but I know of no way to get `pip` to install the right dependencies. 
+
+    pip install osxphotos
+	
 ## Command Line Usage
 
 This package will install a command line utility called `osxphotos` that allows you to query the Photos database.  Alternatively, you can also run the command line utility like this: `python3 -m osxphotos`
@@ -190,6 +201,11 @@ Options:
                                   Search by end item date, e.g.
                                   2000-01-12T12:00:00 or 2000-12-31 (ISO 8601
                                   w/o TZ).
+  --update                        Only export new or updated files. See notes
+                                  below on export and --update.
+  --export-as-hardlink            Hardlink files instead of copying them.
+                                  Cannot be used with --exiftool which creates
+                                  copies of the files with embedded EXIF data.
   --overwrite                     Overwrite existing files. Default behavior
                                   is to add (1), (2), etc to filename if file
                                   already exists. Use this with caution as it
@@ -209,6 +225,22 @@ Options:
                                   photos if the RAW photo does not have an
                                   associated jpeg image (e.g. the RAW file was
                                   imported to Photos without a jpeg preview).
+  --person-keyword                Use person in image as keyword/tag when
+                                  exporting metadata.
+  --album-keyword                 Use album name as keyword/tag when exporting
+                                  metadata.
+  --keyword-template TEMPLATE     For use with --exiftool, --sidecar; specify
+                                  a template string to use as keyword in the
+                                  form '{name,DEFAULT}' This is the same
+                                  format as --directory.  For example, if you
+                                  wanted to add the full path to the folder
+                                  and album photo is contained in as a keyword
+                                  when exporting you could specify --keyword-
+                                  template "{folder_album}" You may specify
+                                  more than one template, for example
+                                  --keyword-template "{folder_album}"
+                                  --keyword-template "{created.year}" See
+                                  Templating System below.
   --current-name                  Use photo's current filename instead of
                                   original filename for export.  Note:
                                   Starting with Photos 5, all photos are
@@ -242,7 +274,8 @@ Options:
                                   exported photos. To use this option,
                                   exiftool must be installed and in the path.
                                   exiftool may be installed from
-                                  https://exiftool.org/
+                                  https://exiftool.org/.  Cannot be used with
+                                  --export-as-hardlink.
   --directory DIRECTORY           Optional template for specifying name of
                                   output directory in the form
                                   '{name,DEFAULT}'. See below for additional
@@ -254,9 +287,37 @@ Options:
                                   get an error while exporting.
   -h, --help                      Show this message and exit.
 
-**Templating System**
+** Export **
+When exporting photos, osxphotos creates a database in the top-level export
+folder called '.osxphotos_export.db'.  This database preserves state
+information used for determining which files need to be updated when run with
+--update.  It is recommended that if you later move the export folder tree you
+also move the database file.
 
-With the --directory option, you may specify a template for the export
+The --update option will only copy new or updated files from the library to
+the export folder.  If a file is changed in the export folder (for example,
+you edited the exported image), osxphotos will detect this as a difference and
+re-export the original image from the library thus overwriting the changes.
+If using --update, the exported library should be treated as a backup, not a
+working copy where you intend to make changes.
+
+Note: The number of files reported for export and the number actually exported
+may differ due to live photos, associated RAW images, and edited photos which
+are reported in the total photos exported.
+
+Implementation note: To determine which files need to be updated, osxphotos
+stores file signature information in the '.osxphotos_export.db' database. The
+signature includes size, modification time, and filename.  In order to
+minimize run time, --update does not do a full comparison (diff) of the files
+nor does it compare hashes of the files.  In normal usage, this is sufficient
+for updating the library. You can always run export without the --update
+option to re-export the entire library thus rebuilding the
+'.osxphotos_export.db' database.
+
+
+** Templating System **
+
+With the --directory option you may specify a template for the export
 directory.  This directory will be appended to the export path specified in
 the export DEST argument to export.  For example, if template is
 '{created.year}/{created.month}', and export desitnation DEST is
@@ -264,6 +325,11 @@ the export DEST argument to export.  For example, if template is
 be '/Users/maria/Pictures/export/2020/March' if the photo was created in March
 2020.
 
+The templating system may also be used with the --keyword-template option to
+set keywords on export (with --exiftool or --sidecar), for example, to set a
+new keyword in format 'folder/subfolder/album' to preserve the folder/album
+structure, you can use --keyword-template "{folder_album}"
+
 In the template, valid template substitutions will be replaced by the
 corresponding value from the table below.  Invalid substitutions will result
 in a an error and the script will abort.
@@ -276,7 +342,7 @@ rendered name, use double braces, e.g. '{{' or '}}', thus using
 You may specify an optional default value to use if the substitution does not
 contain a value (e.g. the value is null) by specifying the default value after
 a ',' in the template string: for example, if template is
-'{created.year}/{place.address,'NO_ADDRESS'}' but there was no address
+'{created.year}/{place.address,NO_ADDRESS}' but there was no address
 associated with the photo, the resulting output would be:
 '2020/NO_ADDRESS/photoname.jpg'. If specified, the default value may not
 contain a brace symbol ('{' or '}').
@@ -288,7 +354,7 @@ I plan to eventually extend the templating system to the exported filename so
 you can specify the filename using a template.
 
 Substitution                    Description
-{name}                          Filename of the photo
+{name}                          Current filename of the photo
 {original_name}                 Photo's original filename when imported to
                                 Photos
 {title}                         Title of the photo
@@ -354,13 +420,16 @@ exported, one to each directory.  For example: --directory
 of the following directories if the photos were created in 2019 and were in
 albums 'Vacation' and 'Family': 2019/Vacation, 2019/Family
 
-Substitution    Description
-{album}         Album(s) photo is contained in
-{folder_album}  Folder path + album photo is contained in. e.g.
-                'Folder/Subfolder/Album' or just 'Album' if no enclosing
-                folder
-{keyword}       Keyword(s) assigned to photo
-{person}        Person(s) / face(s) in a photo
+Substitution        Description
+{album}             Album(s) photo is contained in
+{folder_album}      Folder path + album photo is contained in. e.g.
+                    'Folder/Subfolder/Album' or just 'Album' if no enclosing
+                    folder
+{keyword}           Keyword(s) assigned to photo
+{person}            Person(s) / face(s) in a photo
+{label}             Image categorization label associated with a photo
+                    (Photos 5 only)
+{label_normalized}  All lower case version of 'label' (Photos 5 only)
 ```
 
 Example: export all photos to ~/Desktop/export group in folders by date created
@@ -669,6 +738,29 @@ Returns a dictionary of shared albums (e.g. shared via iCloud photo sharing) fou
 
 **Note**: *Photos 5 / MacOS 10.15 only*.  On earlier versions of Photos, prints warning and returns empty dictionary.
 
+#### `labels`
+Returns image categorization labels associated with photos in the library as list of str.
+
+**Note**: Only valid on Photos 5; on earlier versions, returns empty list. In Photos 5, Photos runs machine learning image categorization against photos in the library and automatically assigns labels to photos such as "People", "Dog", "Water", etc.  A photo may have zero or more labels associated with it.  See also [labels_normalized](#labels_normalized).  
+
+#### `labels_normalized`
+Returns image categorization labels associated with photos in the library as list of str. Labels are normalized (e.g. converted to lower case).  Use of normalized strings makes it easier to search if you don't how Apple capitalizes a label.
+
+**Note**: Only valid on Photos 5; on earlier versions, returns empty list. In Photos 5, Photos runs machine learning image categorization against photos in the library and automatically assigns labels to photos such as "People", "Dog", "Water", etc.  A photo may have zero or more labels associated with it.  See also [labels](#labels).  
+
+#### `labels_as_dict`
+Returns dictionary image categorization labels associated with photos in the library where key is label and value is number of photos in the library with the label.
+
+**Note**: Only valid on Photos 5; on earlier versions, logs warning and returns empty dict. In Photos 5, Photos runs machine learning image categorization against photos in the library and automatically assigns labels to photos such as "People", "Dog", "Water", etc.  A photo may have zero or more labels associated with it.  See also [labels_normalized_as_dict](#labels_normalized_as_dict).  
+
+#### `labels_normalized_as_dict`
+Returns dictionary of image categorization labels associated with photos in the library where key is normalized label and value is number of photos in the library with that label. Labels are normalized (e.g. converted to lower case).  Use of normalized strings makes it easier to search if you don't how Apple capitalizes a label.
+
+**Note**: Only valid on Photos 5; on earlier versions, logs warning and returns empty dict. In Photos 5, Photos runs machine learning image categorization against photos in the library and automatically assigns labels to photos such as "People", "Dog", "Water", etc.  A photo may have zero or more labels associated with it.  See also [labels_as_dict](#labels_as_dict).  
+
+
+
+
 #### `library_path`
 ```python
 # assumes photosdb is a PhotosDB object (see above)
@@ -806,6 +898,7 @@ For example, in my library, Photos says I have 19,386 photos and 474 movies.  Ho
 >>>
 ```
 
+
 ### PhotoInfo 
 PhotosDB.photos() returns a list of PhotoInfo objects.  Each PhotoInfo object represents a single photo in the Photos library.
 
@@ -947,15 +1040,85 @@ Returns True if photo is a panorama, otherwise False.
 
 **Note**: The result of `PhotoInfo.panorama` will differ from the "Panoramas" Media Types smart album in that it will also identify panorama photos from older phones that Photos does not recognize as panoramas. 
 
+#### `labels`
+Returns image categorization labels associated with the photo as list of str.
+
+**Note**: Only valid on Photos 5; on earlier versions, returns empty list. In Photos 5, Photos runs machine learning image categorization against photos in the library and automatically assigns labels to photos such as "People", "Dog", "Water", etc.  A photo may have zero or more labels associated with it.  See also [labels_normalized](#labels_normalized).  
+
+#### `labels_normalized`
+Returns image categorization labels associated with the photo as list of str. Labels are normalized (e.g. converted to lower case).  Use of normalized strings makes it easier to search if you don't how Apple capitalizes a label. For example:
+
+```python
+import osxphotos
+
+photosdb = osxphotos.PhotosDB()
+for photo in photosdb.photos():
+    if "statue" in photo.labels_normalized:
+        print(f"I found a statue! {photo.original_filename}")
+```
+
+**Note**: Only valid on Photos 5; on earlier versions, returns empty list. In Photos 5, Photos runs machine learning image categorization against photos in the library and automatically assigns labels to photos such as "People", "Dog", "Water", etc.  A photo may have zero or more labels associated with it.  See also [labels](#labels).  
+
+#### `exif_info`
+Returns an [ExifInfo](#exifinfo) object with EXIF details from the Photos database.  See [ExifInfo](#exifinfo) for additional details.
+
+**Note**: Only valid on Photos 5; on earlier versions, returns `None`.  The EXIF details returned are a subset of the actual EXIF data in a typical image.  At import Photos stores this subset in the database and it's this stored data that `exif_info` returns.
+
+See also `exiftool`.
+
+#### `exiftool`
+Returns an ExifTool object for the photo which provides an interface to [exiftool](https://exiftool.org/) allowing you to read or write the actual EXIF data in the image file inside the Photos library.  If [exif_info](#exif-info) doesn't give you all the data you need, you can use `exiftool` to read the entire EXIF contents of the image.
+
+If the file is missing from the library (e.g. not downloaded from iCloud), returns None. 
+
+exiftool must be installed in the path for this to work.  If exiftool cannot be found in the path, calling `exiftool` will log a warning and return `None`.  You can check the exiftool path using `osxphotos.exiftool.get_exiftool_path` which will raise FileNotFoundError if exiftool cannot be found.
+
+```python
+>>> import osxphotos
+>>> osxphotos.exiftool.get_exiftool_path()
+'/usr/local/bin/exiftool'
+>>>
+```
+
+`ExifTool` provides the following methods:
+
+- `as_dict()`: returns all EXIF metadata found in the file as a dictionary in following form (Note: this shows just a subset of available metadata).  See [exiftool](https://exiftool.org/) documentation to understand which metadata keys are available.
+```python
+{'Composite:Aperture': 2.2,
+ 'Composite:GPSPosition': '-34.9188916666667 138.596861111111',
+ 'Composite:ImageSize': '2754 2754',
+ 'EXIF:CreateDate': '2017:06:20 17:18:56',
+ 'EXIF:LensMake': 'Apple',
+ 'EXIF:LensModel': 'iPhone 6s back camera 4.15mm f/2.2',
+ 'EXIF:Make': 'Apple',
+ 'XMP:Title': 'Elder Park',
+}
+```
+
+- `json()`: returns same information as `as_dict()` but as a serialized JSON string.
+
+- `setvalue(tag, value)`: write to the EXIF data in the photo file. To delete a tag, use setvalue with value = `None`. For example:
+```python
+photo.exiftool.setvalue("XMP:Title", "Title of photo")
+```
+- `addvalues(tag, *values)`: Add one or more value(s) to tag.  For a tag that accepts multiple values, like "IPTC:Keywords", this will add the values as additional list values.  However, for tags which are not usually lists, such as "EXIF:ISO" this will literally add the new value to the old value which is probably not the desired effect.  Be sure you understand the behavior of the individual tag before using this. For example:
+```python
+photo.exiftool.addvalues("IPTC:Keywords", "vacation", "beach")
+```
+
+**Caution**: I caution against  writing new EXIF data to photos in the Photos library because this will overwrite the original copy of the photo and could adversely affect how Photos behaves.  `exiftool.as_dict()` is useful for getting access to all the photos information but if you want to write new EXIF data, I recommend you export the photo first then write the data.  [PhotoInfo.export()](#export) does this if called with `exiftool=True`.
+
 #### `json()`
 Returns a JSON representation of all photo info 
 
-#### `export(dest, *filename, edited=False, live_photo=False, overwrite=False, increment=True, sidecar_json=False, sidecar_xmp=False, use_photos_export=False, timeout=120, exiftool=False, no_xattr=False)`
+#### `export()`
+`export(dest, *filename, edited=False, live_photo=False, export_as_hardlink=False, overwrite=False, increment=True, sidecar_json=False, sidecar_xmp=False, use_photos_export=False, timeout=120, exiftool=False, no_xattr=False, use_albums_as_keywords=False, use_persons_as_keywords=False)`
 
 Export photo from the Photos library to another destination on disk.  
 - dest: must be valid destination path as str (or exception raised).
 - *filename (optional): name of picture as str; if not provided, will use current filename.  **NOTE**: if provided, user must ensure file extension (suffix) is correct. For example, if photo is .CR2 file, edited image may be .jpeg.  If you provide an extension different than what the actual file is, export will print a warning but will happily export the photo using the incorrect file extension.  e.g. to get the extension of the edited photo, look at [PhotoInfo.path_edited](#path_edited).
 - edited: boolean; if True (default=False), will export the edited version of the photo (or raise exception if no edited version)
+- export_as_hardlink: boolean; if True (default=False), will hardlink files instead of copying them
 - overwrite: boolean; if True (default=False), will overwrite files if they alreay exist
 - live_photo: boolean; if True (default=False), will also export the associted .mov for live photos; exported live photo will be named filename.mov
 - increment: boolean; if True (default=True), will increment file name until a non-existent name is found
@@ -965,8 +1128,10 @@ Export photo from the Photos library to another destination on disk.
 - timeout: (int, default=120) timeout in seconds used with use_photos_export
 - exiftool: (boolean, default = False) if True, will use [exiftool](https://exiftool.org/) to write metadata directly to the exported photo; exiftool must be installed and in the system path
 - no_xattr: (boolean, default = False); if True, exports file without preserving extended attributes
+- use_albums_as_keywords: (boolean, default = False); if True, will use album names as keywords when exporting metadata with exiftool or sidecar
+- use_persons_as_keywords: (boolean, default = False); if True, will use person names as keywords when exporting metadata with exiftool or sidecar
 
-Returns: list of paths to exported files. More than one file could be exported, for example if live_photo=True, both the original imaage and the associated .mov file will be exported
+Returns: list of paths to exported files. More than one file could be exported, for example if live_photo=True, both the original image and the associated .mov file will be exported
 
 The json sidecar file can be used by exiftool to apply the metadata from the json file to the image.  For example: 
 
@@ -986,6 +1151,63 @@ If overwrite=False and increment=False, export will fail if destination file alr
 
 **Implementation Note**: Because the usual python file copy methods don't preserve all the metadata available on MacOS, export uses `/usr/bin/ditto` to do the copy for export. ditto preserves most metadata such as extended attributes, permissions, ACLs, etc.
 
+#### <a name="rendertemplate">`render_template()`</a>
+
+`render_template(template, none_str = "_", path_sep = None)`
+Render template string for photo.  none_str is used if template substitution results in None value and no default specified. 
+- `template`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None.  This is useful for values which are not always present, for example reverse geolocation data.
+- `none_str`: optional str to use as substitution when template value is None and no default specified in the template string.  default is "_".
+- `path_sep`: optional character to use as path separator, default is os.path.sep
+
+Returns a tuple of (rendered, unmatched) where rendered is a list of rendered strings with all substitutions made and unmatched is a list of any strings that resembled a template substitution but did not match a known substitution. E.g. if template contained "{foo}", unmatched would be ["foo"].
+
+e.g. `render_filepath_template("{created.year}/{foo}", photo)` would return `(["2020/{foo}"],["foo"])`
+
+If you want to include "{" or "}" in the output, use "{{" or "}}"
+
+e.g. `render_filepath_template("{created.year}/{{foo}}", photo)` would return `(["2020/{foo}"],[])`
+
+Some substitutions, notably `album`, `keyword`, and `person` could return multiple values, hence a new string will be return for each possible substitution (hence why a list of rendered strings is returned).  For example, a photo in 2 albums: 'Vacation' and 'Family' would result in the following rendered values if template was "{created.year}/{album}" and created.year == 2020: `["2020/Vacation","2020/Family"]` 
+
+See [Template Substitutions](#template-substitutions) for additional details.
+
+### ExifInfo
+[PhotosInfo.exif_info](#exif-info) returns an `ExifInfo` object with some EXIF data about the photo (Photos 5 only).  `ExifInfo` contains the following properties:
+
+```python
+    flash_fired: bool
+    iso: int
+    metering_mode: int
+    sample_rate: int
+    track_format: int
+    white_balance: int
+    aperture: float
+    bit_rate: float
+    duration: float
+    exposure_bias: float
+    focal_length: float
+    fps: float
+    latitude: float
+    longitude: float
+    shutter_speed: float
+    camera_make: str
+    camera_model: str
+    codec: str
+    lens_model: str
+```
+
+For example:
+
+```python
+import osxphotos
+
+nikon_photos = [
+    p
+    for p in osxphotos.PhotosDB().photos()
+    if p.exif_info.camera_make and "nikon" in p.exif_info.camera_make.lower()
+]
+```
+
 ### AlbumInfo
 PhotosDB.album_info and PhotoInfo.album_info return a list of AlbumInfo objects.  Each AlbumInfo object represents a single album in the Photos library.
 
@@ -1131,30 +1353,13 @@ PostalAddress(street='3700 Wailea Alanui Dr', sub_locality=None, city='Kihei', s
 '96753'
 ```
 
-### Template Functions
-
-There is a simple template system used by the command line client to specify the output directory using a template.  The following are available in `osxphotos.template`.
-
-#### `render_filepath_template(template, photo, none_str="_")`
-Render template string for photo.  none_str is used if template substitution results in None value and no default specified. 
-- `template`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None.  This is useful for values which are not always present, for example reverse geolocation data.
-- `photo`: a [PhotoInfo](#photoinfo) object
-- `none_str`: optional str to use as substitution when template value is None and no default specified in the template string.  default is "_".
-
-Returns a tuple of (rendered, unmatched) where rendered is a list of rendered strings with all substitutions made and unmatched is a list of any strings that resembled a template substitution but did not match a known substitution. E.g. if template contained "{foo}", unmatched would be ["foo"].
-
-e.g. `render_filepath_template("{created.year}/{foo}", photo)` would return `(["2020/{foo}"],["foo"])`
-
-If you want to include "{" or "}" in the output, use "{{" or "}}"
-
-e.g. `render_filepath_template("{created.year}/{{foo}}", photo)` would return `(["2020/{foo}"],[])`
-
-Some substitutions, notably `album`, `keyword`, and `person` could return multiple values, hence a new string will be return for each possible substitution (hence why a list of rendered strings is returned).  For example, a photo in 2 albums: 'Vacation' and 'Family' would result in the following rendered values if template was "{created.year}/{album}" and created.year == 2020: `["2020/Vacation","2020/Family"]` 
+### Template Substitutions
 
+The following substitutions are availabe for use with `PhotoInfo.render_template()` 
 
 | Substitution | Description |
 |--------------|-------------|
-|{name}|Filename of the photo|
+|{name}|Current filename of the photo|
 |{original_name}|Photo's original filename when imported to Photos|
 |{title}|Title of the photo|
 |{descr}|Description of the photo|
@@ -1186,25 +1391,13 @@ Some substitutions, notably `album`, `keyword`, and `person` could return multip
 |{place.address.country}|Country name of the postal address, e.g. 'United States'|
 |{place.address.country_code}|ISO country code of the postal address, e.g. 'US'|
 |{album}|Album(s) photo is contained in|
+|{folder_album}|Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder|
 |{keyword}|Keyword(s) assigned to photo|
 |{person}|Person(s) / face(s) in a photo|
+|{label}|Image categorization label associated with a photo (Photos 5 only)|
+|{label_normalized}|All lower case version of 'label' (Photos 5 only)|
 
 
-#### `DateTimeFormatter(dt)`
-Class that provides easy access to formatted datetime values.
-- `dt`: a datetime.datetime object
-
-Returnes `DateTimeFormater` class.
-
-Has the following properties:
-- `date`: Date in ISO format without timezone, e.g. "2020-03-04"
-- `year`: 4-digit year
-- `yy`: 2-digit year
-- `month`: month name in user's locale
-- `mon`: month abbreviation in user's locale
-- `mm`: 2-digit month
-- `doy`: 3-digit day of year (e.g. Julian day)
-
 ### Utility Functions
 
 The following functions are located in osxphotos.utils
@@ -1310,7 +1503,7 @@ Testing against "real world" Photos libraries would be especially helpful.  If y
 
 My goal is make osxphotos as reliable and comprehensive as possible.  The test suite currently has over 400 tests--but there are still some [bugs](https://github.com/RhetTbull/osxphotos/issues?q=is%3Aissue+is%3Aopen+label%3Abug) or incomplete features lurking.  If you find bugs please open an [issue](https://github.com/RhetTbull/osxphotos/issues).  Notable issues include:
 
-- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos.  osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`.  If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored.  See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101) Note: Alpha version of fix for this bug is implemented in the current version of osxphotos.
+- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos.  osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`.  If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored.  See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101) Note: Beta version of fix for this bug is implemented in the current version of osxphotos.
 - The `--download-missing` option for `osxphotos export` does not work correctly with burst images.  It will download the primary image but not the other burst images.  See [Issue #75](https://github.com/RhetTbull/osxphotos/issues/75)
 
 ## Implementation Notes
@@ -1321,6 +1514,8 @@ If apple changes the database format this will likely break.
 
 Apple does provide a framework ([PhotoKit](https://developer.apple.com/documentation/photokit?language=objc)) for querying the user's Photos library and I attempted to create the funcationality in this package using this framework but unfortunately PhotoKit does not provide access to much of the needed metadata (such as Faces/Persons) and Apple's System Integrity Protection (SIP) made the interface unreliable.  If you'd like to experiment with the PhotoKit interface, here's some sample [code](https://gist.github.com/RhetTbull/41cc85e5bdeb30f761147ce32fba5c94).  While copying the sqlite file is a bit kludgy, it allows osxphotos to provide access to all available metadata.
 
+For additional details about how osxphotos is implemented or if you would like to extend the code, see the [wiki](https://github.com/RhetTbull/osxphotos/wiki).
+
 ## Dependencies
 - [PyObjC](https://pythonhosted.org/pyobjc/)
 - [PyYAML](https://pypi.org/project/PyYAML/)

cli.py

@@ -0,0 +1,19 @@
+""" stand alone command line script for use with pyinstaller
+    
+    To build this into an executable:
+    - install pyinstaller:
+        python3 -m pip install pyinstaller
+    - then use make_cli_exe.sh to run pyinstaller or execute the following command:
+        pyinstaller --onefile --hidden-import="pkg_resources.py2_warn" --name osxphotos --add-data osxphotos/templates/xmp_sidecar.mako:osxphotos/templates cli.py
+
+    Resulting executable will be in "dist/osxphotos"
+
+    Note: This is *not* the cli that "python3 -m pip install osxphotos" or "python setup.py install" would install;
+    it's merely a wrapper around __main__.py to allow pyinstaller to work
+    
+"""
+
+from osxphotos.__main__ import cli
+
+if __name__ == "__main__":
+    cli()

make_cli_exe.sh

@@ -0,0 +1,8 @@
+#!/bin/sh 
+
+# This will build an stand-alone executable called 'osxphotos' in your ./dist directory
+# using pyinstaller
+# If you need to install pyinstaller:
+# python3 -m pip install --upgrade pyinstaller
+
+pyinstaller --onefile --hidden-import="pkg_resources.py2_warn" --name osxphotos --add-data osxphotos/templates/xmp_sidecar.mako:osxphotos/templates cli.py

osxphotos/__main__.py

@@ -6,6 +6,7 @@ import os
 import os.path
 import pathlib
 import sys
+import time
 
 import click
 import yaml
@@ -18,15 +19,28 @@ from pathvalidate import (
 
 import osxphotos
 
-from ._constants import _EXIF_TOOL_URL, _PHOTOS_5_VERSION, _UNKNOWN_PLACE
+from ._constants import _EXIF_TOOL_URL, _PHOTOS_4_VERSION, _UNKNOWN_PLACE
 from ._version import __version__
 from .exiftool import get_exiftool_path
-from .template import (
-    render_filepath_template,
+from .photoinfo import ExportResults
+from .photoinfo.template import (
     TEMPLATE_SUBSTITUTIONS,
     TEMPLATE_SUBSTITUTIONS_MULTI_VALUED,
 )
 from .utils import _copy_file, create_path_by_date
+from ._export_db import ExportDB
+
+# global variable to control verbose output
+# set via --verbose/-V
+VERBOSE = False
+
+# name of export DB
+OSXPHOTOS_EXPORT_DB = ".osxphotos_export.db"
+
+
+def verbose(*args, **kwargs):
+    if VERBOSE:
+        click.echo(*args, **kwargs)
 
 
 def get_photos_db(*db_options):
@@ -78,12 +92,44 @@ class ExportCommand(click.Command):
         help_text = super().get_help(ctx)
         formatter = click.HelpFormatter()
 
-        formatter.write("\n\n")
         # passed to click.HelpFormatter.write_dl for formatting
-        formatter.write_text("**Templating System**")
+
+        formatter.write("\n\n")
+        formatter.write_text("** Export **")
+        formatter.write_text(
+            "When exporting photos, osxphotos creates a database in the top-level "
+            + f"export folder called '{OSXPHOTOS_EXPORT_DB}'.  This database preserves state information "
+            + "used for determining which files need to be updated when run with --update.  It is recommended "
+            + "that if you later move the export folder tree you also move the database file."
+        )
         formatter.write("\n")
         formatter.write_text(
-            "With the --directory option, you may specify a template for the "
+            "The --update option will only copy new or updated files from the library "
+            + "to the export folder.  If a file is changed in the export folder (for example, you edited the "
+            + "exported image), osxphotos will detect this as a difference and re-export the original image "
+            + "from the library thus overwriting the changes.  If using --update, the exported library "
+            + "should be treated as a backup, not a working copy where you intend to make changes. "
+        )
+        formatter.write("\n")
+        formatter.write_text("Note: The number of files reported for export and the number actually exported "
+            +"may differ due to live photos, associated RAW images, and edited photos which are reported "
+            +"in the total photos exported.")
+        formatter.write("\n")
+        formatter.write_text(
+            "Implementation note: To determine which files need to be updated, "
+            + f"osxphotos stores file signature information in the '{OSXPHOTOS_EXPORT_DB}' database. "
+            + "The signature includes size, modification time, and filename.  In order to minimize "
+            + "run time, --update does not do a full comparison (diff) of the files nor does it compare "
+            + "hashes of the files.  In normal usage, this is sufficient for updating the library. "
+            + "You can always run export without the --update option to re-export the entire library thus "
+            + f"rebuilding the '{OSXPHOTOS_EXPORT_DB}' database."
+        )
+
+        formatter.write("\n\n")
+        formatter.write_text("** Templating System **")
+        formatter.write("\n")
+        formatter.write_text(
+            "With the --directory option you may specify a template for the "
             + "export directory.  This directory will be appended to the export path specified "
             + "in the export DEST argument to export.  For example, if template is "
             + "'{created.year}/{created.month}', and export desitnation DEST is "
@@ -92,6 +138,13 @@ class ExportCommand(click.Command):
             + "if the photo was created in March 2020. "
         )
         formatter.write("\n")
+        formatter.write_text(
+            "The templating system may also be used with the --keyword-template option "
+            + "to set keywords on export (with --exiftool or --sidecar), "
+            + "for example, to set a new keyword in format 'folder/subfolder/album' to "
+            + 'preserve the folder/album structure, you can use --keyword-template "{folder_album}"'
+        )
+        formatter.write("\n")
         formatter.write_text(
             "In the template, valid template substitutions will be replaced by "
             + "the corresponding value from the table below.  Invalid substitutions will result in a "
@@ -109,7 +162,7 @@ class ExportCommand(click.Command):
             "You may specify an optional default value to use if the substitution does not contain a value "
             + "(e.g. the value is null) "
             + "by specifying the default value after a ',' in the template string: "
-            + "for example, if template is '{created.year}/{place.address,'NO_ADDRESS'}' "
+            + "for example, if template is '{created.year}/{place.address,NO_ADDRESS}' "
             + "but there was no address associated with the photo, the resulting output would be: "
             + "'2020/NO_ADDRESS/photoname.jpg'. "
             + "If specified, the default value may not contain a brace symbol ('{' or '}')."
@@ -428,7 +481,7 @@ def albums(ctx, cli_obj, db, json_, photos_library):
 
     photosdb = osxphotos.PhotosDB(dbfile=db)
     albums = {"albums": photosdb.albums_as_dict}
-    if photosdb.db_version >= _PHOTOS_5_VERSION:
+    if photosdb.db_version > _PHOTOS_4_VERSION:
         albums["shared albums"] = photosdb.albums_shared_as_dict
 
     if json_ or cli_obj.json:
@@ -493,7 +546,7 @@ def info(ctx, cli_obj, db, json_, photos_library):
     not_shared_movies = [p for p in movies if not p.shared]
     info["movie_count"] = len(not_shared_movies)
 
-    if pdb.db_version >= _PHOTOS_5_VERSION:
+    if pdb.db_version > _PHOTOS_4_VERSION:
         shared_photos = [p for p in photos if p.shared]
         info["shared_photo_count"] = len(shared_photos)
 
@@ -508,7 +561,7 @@ def info(ctx, cli_obj, db, json_, photos_library):
     info["albums_count"] = len(albums)
     info["albums"] = albums
 
-    if pdb.db_version >= _PHOTOS_5_VERSION:
+    if pdb.db_version > _PHOTOS_4_VERSION:
         albums_shared = pdb.albums_shared_as_dict
         info["shared_albums_count"] = len(albums_shared)
         info["shared_albums"] = albums_shared
@@ -775,6 +828,7 @@ def query(
     ]
     # print help if no non-exclusive term or a double exclusive term is given
     if not any(nonexclusive + [b ^ n for b, n in exclusive]):
+        click.echo("Incompatible query options", err=True)
         click.echo(cli.commands["query"].get_help(ctx), err=True)
         return
 
@@ -855,8 +909,19 @@ def query(
 
 @cli.command(cls=ExportCommand)
 @DB_OPTION
-@click.option("--verbose", "-V", is_flag=True, help="Print verbose output.")
+@click.option("--verbose", "-V", "verbose_", is_flag=True, help="Print verbose output.")
 @query_options
+@click.option(
+    "--update",
+    is_flag=True,
+    help="Only export new or updated files. See notes below on export and --update.",
+)
+@click.option(
+    "--export-as-hardlink",
+    is_flag=True,
+    help="Hardlink files instead of copying them.  "
+    "Cannot be used with --exiftool which creates copies of the files with embedded EXIF data.",
+)
 @click.option(
     "--overwrite",
     is_flag=True,
@@ -893,6 +958,30 @@ def query(
     "Note: this does not skip RAW photos if the RAW photo does not have an associated jpeg image "
     "(e.g. the RAW file was imported to Photos without a jpeg preview).",
 )
+@click.option(
+    "--person-keyword",
+    is_flag=True,
+    help="Use person in image as keyword/tag when exporting metadata.",
+)
+@click.option(
+    "--album-keyword",
+    is_flag=True,
+    help="Use album name as keyword/tag when exporting metadata.",
+)
+@click.option(
+    "--keyword-template",
+    metavar="TEMPLATE",
+    multiple=True,
+    default=None,
+    help="For use with --exiftool, --sidecar; specify a template string to use as "
+    "keyword in the form '{name,DEFAULT}' "
+    "This is the same format as --directory.  For example, if you wanted to add "
+    "the full path to the folder and album photo is contained in as a keyword when exporting "
+    'you could specify --keyword-template "{folder_album}" '
+    'You may specify more than one template, for example --keyword-template "{folder_album}" '
+    '--keyword-template "{created.year}" '
+    "See Templating System below.",
+)
 @click.option(
     "--current-name",
     is_flag=True,
@@ -929,7 +1018,8 @@ def query(
     is_flag=True,
     help="Use exiftool to write metadata directly to exported photos. "
     "To use this option, exiftool must be installed and in the path.  "
-    "exiftool may be installed from https://exiftool.org/",
+    "exiftool may be installed from https://exiftool.org/.  "
+    "Cannot be used with --export-as-hardlink.",
 )
 @click.option(
     "--directory",
@@ -976,13 +1066,18 @@ def export(
     not_shared,
     from_date,
     to_date,
-    verbose,
+    verbose_,
+    update,
+    export_as_hardlink,
     overwrite,
     export_by_date,
     skip_edited,
     skip_bursts,
     skip_live,
     skip_raw,
+    person_keyword,
+    album_keyword,
+    keyword_template,
     current_name,
     sidecar,
     only_photos,
@@ -1026,6 +1121,9 @@ def export(
         to modify this behavior. 
     """
 
+    global VERBOSE
+    VERBOSE = True if verbose_ else False
+
     if not os.path.isdir(dest):
         sys.exit("DEST must be valid path")
 
@@ -1046,9 +1144,11 @@ def export(
         (selfie, not_selfie),
         (panorama, not_panorama),
         (export_by_date, directory),
+        (export_as_hardlink, exiftool),
         (any(place), no_place),
     ]
     if any([all(bb) for bb in exclusive]):
+        click.echo("Incompatible export options", err=True)
         click.echo(cli.commands["export"].get_help(ctx), err=True)
         return
 
@@ -1090,6 +1190,9 @@ def export(
         _list_libraries()
         return
 
+    # open export database
+    export_db = ExportDB(os.path.join(dest, OSXPHOTOS_EXPORT_DB))
+
     photos = _query(
         db=db,
         keyword=keyword,
@@ -1144,6 +1247,11 @@ def export(
         no_place=no_place,
     )
 
+    results_exported = []
+    results_new = []
+    results_updated = []
+    results_skipped = []
+    results_exif_updated = []
     if photos:
         if export_bursts:
             # add the burst_photos to the export set
@@ -1155,16 +1263,19 @@ def export(
         num_photos = len(photos)
         photo_str = "photos" if num_photos > 1 else "photo"
         click.echo(f"Exporting {num_photos} {photo_str} to {dest}...")
-        if not verbose:
+        start_time = time.perf_counter()
+        if not verbose_:
             # show progress bar
             with click.progressbar(photos) as bar:
                 for p in bar:
-                    export_photo(
+                    results = export_photo(
                         p,
                         dest,
-                        verbose,
+                        verbose_,
                         export_by_date,
                         sidecar,
+                        update,
+                        export_as_hardlink,
                         overwrite,
                         export_edited,
                         original_name,
@@ -1174,15 +1285,26 @@ def export(
                         directory,
                         no_extended_attributes,
                         export_raw,
+                        album_keyword,
+                        person_keyword,
+                        keyword_template,
+                        export_db,
                     )
+                    results_exported.extend(results.exported)
+                    results_new.extend(results.new)
+                    results_updated.extend(results.updated)
+                    results_skipped.extend(results.skipped)
+                    results_exif_updated.extend(results.exif_updated)
         else:
             for p in photos:
-                export_paths = export_photo(
+                results = export_photo(
                     p,
                     dest,
-                    verbose,
+                    verbose_,
                     export_by_date,
                     sidecar,
+                    update,
+                    export_as_hardlink,
                     overwrite,
                     export_edited,
                     original_name,
@@ -1192,14 +1314,43 @@ def export(
                     directory,
                     no_extended_attributes,
                     export_raw,
+                    album_keyword,
+                    person_keyword,
+                    keyword_template,
+                    export_db,
                 )
-                if export_paths:
-                    click.echo(f"Exported {p.filename} to {export_paths}")
-                else:
-                    click.echo(f"Did not export missing file {p.filename}")
+                results_exported.extend(results.exported)
+                results_new.extend(results.new)
+                results_updated.extend(results.updated)
+                results_skipped.extend(results.skipped)
+                results_exif_updated.extend(results.exif_updated)
+
+        stop_time = time.perf_counter()
+        # print summary results
+        if not update:
+            photo_str = "photos" if len(results_exported) != 1 else "photo"
+            click.echo(f"Exported: {len(results_exported)} {photo_str}")
+            click.echo(f"Elapsed time: {stop_time-start_time} seconds")
+        else:
+            photo_str_new = "photos" if len(results_new) != 1 else "photo"
+            photo_str_updated = "photos" if len(results_new) != 1 else "photo"
+            photo_str_skipped = "photos" if len(results_skipped) != 1 else "photo"
+            photo_str_exif_updated = (
+                "photos" if len(results_exif_updated) != 1 else "photo"
+            )
+            click.echo(
+                f"Exported: {len(results_new)} {photo_str_new}, "
+                + f"updated: {len(results_updated)} {photo_str_updated}, "
+                + f"skipped: {len(results_skipped)} {photo_str_skipped}, "
+                + f"updated EXIF data: {len(results_exif_updated)} {photo_str_exif_updated}"
+            )
+            click.echo(f"Elapsed time: {stop_time-start_time} seconds")
+
     else:
         click.echo("Did not find any photos to export")
 
+    export_db.close()
+
 
 @cli.command()
 @click.argument("topic", default=None, required=False, nargs=1)
@@ -1566,9 +1717,11 @@ def _query(
 def export_photo(
     photo,
     dest,
-    verbose,
+    verbose_,
     export_by_date,
     sidecar,
+    update,
+    export_as_hardlink,
     overwrite,
     export_edited,
     original_name,
@@ -1578,13 +1731,18 @@ def export_photo(
     directory,
     no_extended_attributes,
     export_raw,
+    album_keyword,
+    person_keyword,
+    keyword_template,
+    export_db,
 ):
     """ Helper function for export that does the actual export
         photo: PhotoInfo object
         dest: destination path as string
-        verbose: boolean; print verbose output
+        verbose_: boolean; print verbose output
         export_by_date: boolean; create export folder in form dest/YYYY/MM/DD
         sidecar: list zero, 1 or 2 of ["json","xmp"] of sidecar variety to export
+        export_as_hardlink: boolean; hardlink files instead of copying them
         overwrite: boolean; overwrite dest file if it already exists
         original_name: boolean; use original filename instead of current filename
         export_live: boolean; also export live video component if photo is a live photo
@@ -1594,8 +1752,13 @@ def export_photo(
         directory: template used to determine output directory
         no_extended_attributes: boolean; if True, exports photo without preserving extended attributes
         export_raw: boolean; if True exports RAW image associate with the photo
+        album_keyword: boolean; if True, exports album names as keywords in metadata
+        person_keyword: boolean; if True, exports person names as keywords in metadata
+        keyword_template: list of strings; if provided use rendered template strings as keywords
         returns list of path(s) of exported photo or None if photo was missing 
     """
+    global VERBOSE
+    VERBOSE = True if verbose_ else False
 
     # Can export to multiple paths
     # Start with single path [dest] but direcotry and export_by_date will modify dest_paths
@@ -1603,21 +1766,21 @@ def export_photo(
 
     if not download_missing:
         if photo.ismissing:
-            space = " " if not verbose else ""
-            click.echo(f"{space}Skipping missing photo {photo.filename}")
-            return None
+            space = " " if not verbose_ else ""
+            verbose(f"{space}Skipping missing photo {photo.filename}")
+            return ExportResults([], [], [], [], [])
         elif not os.path.exists(photo.path):
-            space = " " if not verbose else ""
-            click.echo(
+            space = " " if not verbose_ else ""
+            verbose(
                 f"{space}WARNING: file {photo.path} is missing but ismissing=False, "
                 f"skipping {photo.filename}"
             )
-            return None
+            return ExportResults([], [], [], [], [])
     elif photo.ismissing and not photo.iscloudasset or not photo.incloud:
-        click.echo(
+        verbose(
             f"Skipping missing {photo.filename}: not iCloud asset or missing from cloud"
         )
-        return None
+        return ExportResults([], [], [], [], [])
 
     filename = None
     if original_name:
@@ -1625,8 +1788,7 @@ def export_photo(
     else:
         filename = photo.filename
 
-    if verbose:
-        click.echo(f"Exporting {photo.filename} as {filename}")
+    verbose(f"Exporting {photo.filename} as {filename}")
 
     if export_by_date:
         date_created = photo.date.timetuple()
@@ -1634,7 +1796,7 @@ def export_photo(
         dest_paths = [dest_path]
     elif directory:
         # got a directory template, render it and check results are valid
-        dirnames, unmatched = render_filepath_template(directory, photo)
+        dirnames, unmatched = photo.render_template(directory)
         if unmatched:
             raise click.BadOptionUsage(
                 "directory",
@@ -1643,9 +1805,9 @@ def export_photo(
         dest_paths = []
         for dirname in dirnames:
             dirname = sanitize_filepath(dirname, platform="auto")
-            if not is_valid_filepath(dirname, platform="auto"):
-                raise ValueError(f"Invalid file path: {dirname}")
             dest_path = os.path.join(dest, dirname)
+            if not is_valid_filepath(dest_path, platform="auto"):
+                raise ValueError(f"Invalid file path: '{dest_path}'")
             if not os.path.isdir(dest_path):
                 os.makedirs(dest_path)
             dest_paths.append(dest_path)
@@ -1664,21 +1826,46 @@ def export_photo(
     )
 
     # export the photo to each path in dest_paths
-    photo_paths = []
+    results_exported = []
+    results_new = []
+    results_updated = []
+    results_skipped = []
+    results_exif_updated = []
     for dest_path in dest_paths:
-        photo_path = photo.export(
+        export_results = photo.export2(
             dest_path,
             filename,
             sidecar_json=sidecar_json,
             sidecar_xmp=sidecar_xmp,
             live_photo=export_live,
             raw_photo=export_raw,
+            export_as_hardlink=export_as_hardlink,
             overwrite=overwrite,
             use_photos_export=use_photos_export,
             exiftool=exiftool,
             no_xattr=no_extended_attributes,
-        )[0]
-        photo_paths.append(photo_path)
+            use_albums_as_keywords=album_keyword,
+            use_persons_as_keywords=person_keyword,
+            keyword_template=keyword_template,
+            update=update,
+            export_db=export_db,
+        )
+
+        results_exported.extend(export_results.exported)
+        results_new.extend(export_results.new)
+        results_updated.extend(export_results.updated)
+        results_skipped.extend(export_results.skipped)
+        results_exif_updated.extend(export_results.exif_updated)
+
+        if verbose_:
+            for exported in export_results.exported:
+                verbose(f"Exported {exported}")
+            for new in export_results.new:
+                verbose(f"Exported new file {new}")
+            for updated in export_results.updated:
+                verbose(f"Exported updated file {updated}")
+            for skipped in export_results.skipped:
+                verbose(f"Skipped up to date file {skipped}")
 
         # if export-edited, also export the edited version
         # verify the photo has adjustments and valid path to avoid raising an exception
@@ -1687,7 +1874,7 @@ def export_photo(
             # try to download with Photos
             use_photos_export = download_missing and photo.path_edited is None
             if not download_missing and photo.path_edited is None:
-                click.echo(f"Skipping missing edited photo for {filename}")
+                verbose(f"Skipping missing edited photo for {filename}")
             else:
                 edited_name = pathlib.Path(filename)
                 # check for correct edited suffix
@@ -1698,23 +1885,48 @@ def export_photo(
                     # will be corrected by use_photos_export
                     edited_suffix = pathlib.Path(photo.filename).suffix
                 edited_name = f"{edited_name.stem}_edited{edited_suffix}"
-                if verbose:
-                    click.echo(
-                        f"Exporting edited version of {filename} as {edited_name}"
-                    )
-                photo.export(
+                verbose(f"Exporting edited version of {filename} as {edited_name}")
+                export_results_edited = photo.export2(
                     dest_path,
                     edited_name,
                     sidecar_json=sidecar_json,
                     sidecar_xmp=sidecar_xmp,
+                    export_as_hardlink=export_as_hardlink,
                     overwrite=overwrite,
                     edited=True,
                     use_photos_export=use_photos_export,
                     exiftool=exiftool,
                     no_xattr=no_extended_attributes,
+                    use_albums_as_keywords=album_keyword,
+                    use_persons_as_keywords=person_keyword,
+                    keyword_template=keyword_template,
+                    update=update,
+                    export_db=export_db,
                 )
 
-    return photo_paths
+                results_exported.extend(export_results_edited.exported)
+                results_new.extend(export_results_edited.new)
+                results_updated.extend(export_results_edited.updated)
+                results_skipped.extend(export_results_edited.skipped)
+                results_exif_updated.extend(export_results_edited.exif_updated)
+
+                if verbose_:
+                    for exported in export_results_edited.exported:
+                        verbose(f"Exported {exported}")
+                    for new in export_results_edited.new:
+                        verbose(f"Exported new file {new}")
+                    for updated in export_results_edited.updated:
+                        verbose(f"Exported updated file {updated}")
+                    for skipped in export_results_edited.skipped:
+                        verbose(f"Skipped up to date file {skipped}")
+
+    return ExportResults(
+        results_exported,
+        results_new,
+        results_updated,
+        results_skipped,
+        results_exif_updated,
+    )
 
 
 if __name__ == "__main__":

osxphotos/_constants.py

@@ -51,3 +51,15 @@ _PHOTOS_5_ROOT_FOLDER_KIND = 3999  # root folder
 _PHOTOS_4_ALBUM_KIND = 3  # RKAlbum.albumSubclass
 _PHOTOS_4_TOP_LEVEL_ALBUM = "TopLevelAlbums"
 _PHOTOS_4_ROOT_FOLDER = "LibraryFolder"
+
+# EXIF related constants
+# max keyword length for IPTC:Keyword, reference
+# https://www.iptc.org/std/photometadata/documentation/userguide/
+_MAX_IPTC_KEYWORD_LEN = 64
+
+# Sentinel value for detecting if a template in keyword_template doesn't match
+# If anyone has a keyword matching this, then too bad...
+_OSXPHOTOS_NONE_SENTINEL = "OSXPhotosXYZZY42_Sentinel$"
+
+# SearchInfo categories for Photos 5, corresponds to categories in database/search/psi.sqlite
+SEARCH_CATEGORY_LABEL = 2024

osxphotos/_export_db.py

@@ -0,0 +1,439 @@
+""" Helper class for managing a database used by 
+    PhotoInfo.export for tracking state of exports and updates
+"""
+
+import datetime
+import logging
+import os
+import pathlib
+import sqlite3
+import sys
+from abc import ABC, abstractmethod
+from sqlite3 import Error
+
+from ._version import __version__
+
+OSXPHOTOS_EXPORTDB_VERSION = "1.0"
+
+
+class ExportDB_ABC(ABC):
+    @abstractmethod
+    def get_uuid_for_file(self, filename):
+        pass
+
+    @abstractmethod
+    def set_uuid_for_file(self, filename, uuid):
+        pass
+
+    @abstractmethod
+    def set_stat_orig_for_file(self, filename, stats):
+        pass
+
+    @abstractmethod
+    def get_stat_orig_for_file(self, filename):
+        pass
+
+    @abstractmethod
+    def set_stat_exif_for_file(self, filename, stats):
+        pass
+
+    @abstractmethod
+    def get_stat_exif_for_file(self, filename):
+        pass
+
+    @abstractmethod
+    def get_info_for_uuid(self, uuid):
+        pass
+
+    @abstractmethod
+    def set_info_for_uuid(self, uuid, info):
+        pass
+
+    @abstractmethod
+    def get_exifdata_for_file(self, uuid):
+        pass
+
+    @abstractmethod
+    def set_exifdata_for_file(self, uuid, exifdata):
+        pass
+
+    @abstractmethod
+    def set_data(self, file, uuid, orig_stat, exif_stat, info_json, exif_json):
+        pass
+
+
+class ExportDBNoOp(ExportDB_ABC):
+    """ An ExportDB with NoOp methods """
+
+    def get_uuid_for_file(self, filename):
+        pass
+
+    def set_uuid_for_file(self, filename, uuid):
+        pass
+
+    def set_stat_orig_for_file(self, filename, stats):
+        pass
+
+    def get_stat_orig_for_file(self, filename):
+        pass
+
+    def set_stat_exif_for_file(self, filename, stats):
+        pass
+
+    def get_stat_exif_for_file(self, filename):
+        pass
+
+    def get_info_for_uuid(self, uuid):
+        pass
+
+    def set_info_for_uuid(self, uuid, info):
+        pass
+
+    def get_exifdata_for_file(self, uuid):
+        pass
+
+    def set_exifdata_for_file(self, uuid, exifdata):
+        pass
+
+    def set_data(self, file, uuid, orig_stat, exif_stat, info_json, exif_json):
+        pass
+
+
+class ExportDB(ExportDB_ABC):
+    """ Interface to sqlite3 database used to store state information for osxphotos export command """
+
+    def __init__(self, dbfile):
+        """ dbfile: path to osxphotos export database file """
+        self._dbfile = dbfile
+        # _path is parent of the database
+        # all files referenced by get_/set_uuid_for_file will be converted to
+        # relative paths to this parent _path
+        # this allows the entire export tree to be moved to a new disk/location
+        # whilst preserving the UUID to filename mappping
+        self._path = pathlib.Path(dbfile).parent
+        self._conn = self._open_export_db(dbfile)
+        self._insert_run_info()
+
+    def get_uuid_for_file(self, filename):
+        """ query database for filename and return UUID
+            returns None if filename not found in database
+        """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        logging.debug(f"get_uuid: {filename}")
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                f"SELECT uuid FROM files WHERE filepath_normalized = ?", (filename,)
+            )
+            results = c.fetchone()
+            uuid = results[0] if results else None
+        except Error as e:
+            logging.warning(e)
+            uuid = None
+
+        logging.debug(f"get_uuid: {uuid}")
+        return uuid
+
+    def set_uuid_for_file(self, filename, uuid):
+        """ set UUID of filename to uuid in the database """
+        filename = str(pathlib.Path(filename).relative_to(self._path))
+        filename_normalized = filename.lower()
+        logging.debug(f"set_uuid: {filename} {uuid}")
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                f"INSERT OR REPLACE INTO files(filepath, filepath_normalized, uuid) VALUES (?, ?, ?);",
+                (filename, filename_normalized, uuid),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)
+
+    def set_stat_orig_for_file(self, filename, stats):
+        """ set stat info for filename
+            filename: filename to set the stat info for
+            stat: a tuple of length 3: mode, size, mtime """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        if len(stats) != 3:
+            raise ValueError(f"expected 3 elements for stat, got {len(stats)}")
+
+        logging.debug(f"set_stat_orig_for_file: {filename} {stats}")
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "UPDATE files "
+                + "SET orig_mode = ?, orig_size = ?, orig_mtime = ? "
+                + "WHERE filepath_normalized = ?;",
+                (*stats, filename),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)
+
+    def get_stat_orig_for_file(self, filename):
+        """ get stat info for filename
+            returns: tuple of (mode, size, mtime) 
+        """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "SELECT orig_mode, orig_size, orig_mtime FROM files WHERE filepath_normalized = ?",
+                (filename,),
+            )
+            results = c.fetchone()
+            stats = results[0:3] if results else None
+        except Error as e:
+            logging.warning(e)
+            stats = (None, None, None)
+
+        logging.debug(f"get_stat_orig_for_file: {stats}")
+        return stats
+
+    def set_stat_exif_for_file(self, filename, stats):
+        """ set stat info for filename (after exiftool has updated it)
+            filename: filename to set the stat info for
+            stat: a tuple of length 3: mode, size, mtime """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        if len(stats) != 3:
+            raise ValueError(f"expected 3 elements for stat, got {len(stats)}")
+
+        logging.debug(f"set_stat_exif_for_file: {filename} {stats}")
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "UPDATE files "
+                + "SET exif_mode = ?, exif_size = ?, exif_mtime = ? "
+                + "WHERE filepath_normalized = ?;",
+                (*stats, filename),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)
+
+    def get_stat_exif_for_file(self, filename):
+        """ get stat info for filename (after exiftool has updated it)
+            returns: tuple of (mode, size, mtime) 
+        """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "SELECT exif_mode, exif_size, exif_mtime FROM files WHERE filepath_normalized = ?",
+                (filename,),
+            )
+            results = c.fetchone()
+            stats = results[0:3] if results else None
+        except Error as e:
+            logging.warning(e)
+            stats = (None, None, None)
+
+        logging.debug(f"get_stat_exif_for_file: {stats}")
+        return stats
+
+    def get_info_for_uuid(self, uuid):
+        """ returns the info JSON struct for a UUID """
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute("SELECT json_info FROM info WHERE uuid = ?", (uuid,))
+            results = c.fetchone()
+            info = results[0] if results else None
+        except Error as e:
+            logging.warning(e)
+            info = None
+
+        logging.debug(f"get_info: {uuid}, {info}")
+        return info
+
+    def set_info_for_uuid(self, uuid, info):
+        """ sets the info JSON struct for a UUID """
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "INSERT OR REPLACE INTO info(uuid, json_info) VALUES (?, ?);",
+                (uuid, info),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)
+
+        logging.debug(f"set_info: {uuid}, {info}")
+
+    def get_exifdata_for_file(self, filename):
+        """ returns the exifdata JSON struct for a file """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "SELECT json_exifdata FROM exifdata WHERE filepath_normalized = ?",
+                (filename,),
+            )
+            results = c.fetchone()
+            exifdata = results[0] if results else None
+        except Error as e:
+            logging.warning(e)
+            exifdata = None
+
+        logging.debug(f"get_exifdata: {filename}, {exifdata}")
+        return exifdata
+
+    def set_exifdata_for_file(self, filename, exifdata):
+        """ sets the exifdata JSON struct for a file """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                "INSERT OR REPLACE INTO exifdata(filepath_normalized, json_exifdata) VALUES (?, ?);",
+                (filename, exifdata),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)
+
+        logging.debug(f"set_exifdata: {filename}, {exifdata}")
+
+    def set_data(self, file, uuid, orig_stat, exif_stat, info_json, exif_json):
+        """ sets all the data for file and uuid at once 
+            calls set_uuid_for_file
+                  set_info_for_uuid
+                  set_stat_orig_for_file
+                  set_stat_exif_for_file
+                  set_exifdata_for_file        
+        """
+        filename = str(pathlib.Path(filename).relative_to(self._path)).lower()
+
+        self.set_uuid_for_file(filename, uuid)
+        self.set_info_for_uuid(uuid, info_json)
+        self.set_stat_orig_for_file(filename, orig_stat)
+        self.set_stat_exif_for_file(filename, exif_stat)
+        self.set_exifdata_for_file(filename, exif_json)
+
+    def close(self):
+        """ close the database connection """
+        try:
+            self._conn.close()
+        except Error as e:
+            logging.warning(e)
+
+    def _open_export_db(self, dbfile):
+        """ open export database and return a db connection
+            if dbfile does not exist, will create and initialize the database 
+            returns: connection to the database 
+        """
+
+        if not os.path.isfile(dbfile):
+            logging.debug(f"dbfile {dbfile} doesn't exist, creating it")
+            conn = self._get_db_connection(dbfile)
+            if conn:
+                self._create_db_tables(conn)
+            else:
+                raise Exception("Error getting connection to database {dbfile}")
+        else:
+            logging.debug(f"dbfile {dbfile} exists, opening it")
+            conn = self._get_db_connection(dbfile)
+
+        return conn
+
+    def _get_db_connection(self, dbfile):
+        """ return db connection to dbname """
+        try:
+            conn = sqlite3.connect(dbfile)
+        except Error as e:
+            logging.warning(e)
+            conn = None
+
+        return conn
+
+    def _create_db_tables(self, conn):
+        """ create (if not already created) the necessary db tables for the export database
+            conn: sqlite3 db connection 
+        """
+        sql_commands = {
+            "sql_version_table": """ CREATE TABLE IF NOT EXISTS version (
+                                id INTEGER PRIMARY KEY,
+                                osxphotos TEXT,
+                                exportdb TEXT 
+                                ); """,
+            "sql_files_table": """ CREATE TABLE IF NOT EXISTS files (
+                              id INTEGER PRIMARY KEY,
+                              filepath TEXT NOT NULL,
+                              filepath_normalized TEXT NOT NULL,
+                              uuid TEXT,
+                              orig_mode INTEGER,
+                              orig_size INTEGER,
+                              orig_mtime REAL,
+                              exif_mode INTEGER,
+                              exif_size INTEGER,
+                              exif_mtime REAL
+                              ); """,
+            "sql_runs_table": """ CREATE TABLE IF NOT EXISTS runs (
+                             id INTEGER PRIMARY KEY,
+                             datetime TEXT,
+                             python_path TEXT,
+                             script_name TEXT,
+                             args TEXT,
+                             cwd TEXT 
+                             ); """,
+            "sql_info_table": """ CREATE TABLE IF NOT EXISTS info (
+                             id INTEGER PRIMARY KEY,
+                             uuid text NOT NULL,
+                             json_info JSON 
+                             ); """,
+            "sql_exifdata_table": """ CREATE TABLE IF NOT EXISTS exifdata (
+                             id INTEGER PRIMARY KEY,
+                             filepath_normalized TEXT NOT NULL,
+                             json_exifdata JSON 
+                             ); """,
+            "sql_files_idx": """ CREATE UNIQUE INDEX idx_files_filepath_normalized on files (filepath_normalized); """,
+            "sql_info_idx": """ CREATE UNIQUE INDEX idx_info_uuid on info (uuid); """,
+            "sql_exifdata_idx": """ CREATE UNIQUE INDEX idx_exifdata_filename on exifdata (filepath_normalized); """,
+        }
+        try:
+            c = conn.cursor()
+            for cmd in sql_commands.values():
+                c.execute(cmd)
+            c.execute(
+                "INSERT INTO version(osxphotos, exportdb) VALUES (?, ?);",
+                (__version__, OSXPHOTOS_EXPORTDB_VERSION),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)
+
+    def __del__(self):
+        """ ensure the database connection is closed """
+        if self._conn:
+            try:
+                self._conn.close()
+            except Error as e:
+                logging.warning(e)
+
+    def _insert_run_info(self):
+        dt = datetime.datetime.utcnow().isoformat()
+        python_path = sys.executable
+        cmd = sys.argv[0]
+        if len(sys.argv) > 1:
+            args = " ".join(sys.argv[1:])
+        else:
+            args = ""
+        cwd = os.getcwd()
+        conn = self._conn
+        try:
+            c = conn.cursor()
+            c.execute(
+                f"INSERT INTO runs (datetime, python_path, script_name, args, cwd) VALUES (?, ?, ?, ?, ?)",
+                (dt, python_path, cmd, args, cwd),
+            )
+            conn.commit()
+        except Error as e:
+            logging.warning(e)

osxphotos/_filecmp.py

@@ -0,0 +1,54 @@
+"""Utilities for comparing files 
+
+Modified from CPython/Lib/filecmp.py
+
+Functions:
+    cmp_file(f1, s2) -> int
+    file_sig(f1) -> Tuple[int, int, float]
+
+"""
+
+import os
+import stat
+
+__all__ = ["cmp", "sig"]
+
+
+def cmp_file(f1, s2):
+    """Compare file f1 to signature s2.
+
+    Arguments:
+
+    f1 --  File name
+
+    s2 -- stats as returned by sig
+
+    Return value:
+
+    True if the files are the same, False otherwise.
+
+    This function uses a cache for past comparisons and the results,
+    with cache entries invalidated if their stat information
+    changes.  The cache may be cleared by calling clear_cache().
+
+    """
+
+    if not s2:
+        return False
+
+    s1 = _sig(os.stat(f1))
+
+    if s1[0] != stat.S_IFREG or s2[0] != stat.S_IFREG:
+        return False
+    if s1 == s2:
+        return True
+    return False
+
+
+def _sig(st):
+    return (stat.S_IFMT(st.st_mode), st.st_size, st.st_mtime)
+
+
+def file_sig(f1):
+    """ return os.stat signature for file f1 """
+    return _sig(os.stat(f1))

osxphotos/_version.py

@@ -1,3 +1,3 @@
 """ version info """
 
-__version__ = "0.28.2"
+__version__ = "0.29.2"

osxphotos/datetime_formatter.py

@@ -0,0 +1,52 @@
+""" Simple formatting of datetime.datetime objects """
+
+import datetime
+
+
+class DateTimeFormatter:
+    """ provides property access to formatted datetime.datetime strftime values """
+
+    def __init__(self, dt: datetime.datetime):
+        self.dt = dt
+
+    @property
+    def date(self):
+        """ ISO date in form 2020-03-22 """
+        date = self.dt.date().isoformat()
+        return date
+
+    @property
+    def year(self):
+        """ 4 digit year """
+        year = f"{self.dt.year}"
+        return year
+
+    @property
+    def yy(self):
+        """ 2 digit year """
+        yy = f"{self.dt.strftime('%y')}"
+        return yy
+
+    @property
+    def mm(self):
+        """ 2 digit month """
+        mm = f"{self.dt.strftime('%m')}"
+        return mm
+
+    @property
+    def month(self):
+        """ Month as locale's full name """
+        month = f"{self.dt.strftime('%B')}"
+        return month
+
+    @property
+    def mon(self):
+        """ Month as locale's abbreviated name """
+        mon = f"{self.dt.strftime('%b')}"
+        return mon
+
+    @property
+    def doy(self):
+        """ Julian day of year starting from 001 """
+        doy = f"{self.dt.strftime('%j')}"
+        return doy

osxphotos/exiftool.py

@@ -10,7 +10,7 @@ import logging
 import os
 import subprocess
 import sys
-from functools import lru_cache # pylint: disable=syntax-error
+from functools import lru_cache  # pylint: disable=syntax-error
 
 from .utils import _debug
 
@@ -232,20 +232,27 @@ class ExifTool:
         ver = self.run_commands("-ver", no_file=True)
         return ver.decode("utf-8")
 
-    def json(self):
-        """ return JSON dictionary from exiftool as dict """
+    def as_dict(self):
+        """ return dictionary of all EXIF tags and values from exiftool 
+            returns empty dict if no tags
+        """
         json_str = self.run_commands("-json")
         if json_str:
-            return json.loads(json_str)
+            exifdict = json.loads(json_str)
+            return exifdict[0]
         else:
-            return None
+            return dict()
+
+    def json(self):
+        """ returns JSON string containing all EXIF tags and values from exiftool """
+        json_str = self.run_commands("-json")
+        return json_str
 
     def _read_exif(self):
         """ read exif data from file """
-        json = self.json()
-        self.data = {k: v for k, v in json[0].items()}
+        data = self.as_dict()
+        self.data = {k: v for k, v in data.items()}
 
     def __str__(self):
         str_ = f"file: {self.file}\nexiftool: {self._exiftoolproc._exiftool}"
         return str_
-

osxphotos/photoinfo/__init__.py

@@ -0,0 +1,9 @@
+"""
+PhotoInfo class
+Represents a single photo in the Photos library and provides access to the photo's attributes
+PhotosDB.photos() returns a list of PhotoInfo objects
+"""
+
+from ._photoinfo_exifinfo import ExifInfo
+from ._photoinfo_export import ExportResults
+from .photoinfo import PhotoInfo

osxphotos/photoinfo/_photoinfo_exifinfo.py

@@ -0,0 +1,94 @@
+""" PhotoInfo methods to expose EXIF info from the library """
+
+import logging
+from dataclasses import dataclass
+
+from .._constants import _PHOTOS_4_VERSION
+
+
+@dataclass(frozen=True)
+class ExifInfo:
+    """ EXIF info associated with a photo from the Photos library """
+
+    flash_fired: bool
+    iso: int
+    metering_mode: int
+    sample_rate: int
+    track_format: int
+    white_balance: int
+    aperture: float
+    bit_rate: float
+    duration: float
+    exposure_bias: float
+    focal_length: float
+    fps: float
+    latitude: float
+    longitude: float
+    shutter_speed: float
+    camera_make: str
+    camera_model: str
+    codec: str
+    lens_model: str
+
+
+@property
+def exif_info(self):
+    """ Returns an ExifInfo object with the EXIF data for photo
+        Note: the returned EXIF data is the data Photos stores in the database on import;
+        ExifInfo does not provide access to the EXIF info in the actual image file
+        Some or all of the fields may be None
+        Only valid for Photos 5; on earlier database returns None
+        """
+
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        logging.debug(f"exif_info not implemented for this database version")
+        return None
+
+    try:
+        exif = self._db._db_exifinfo_uuid[self.uuid]
+        exif_info = ExifInfo(
+            iso=exif["ZISO"],
+            flash_fired=True if exif["ZFLASHFIRED"] == 1 else False,
+            metering_mode=exif["ZMETERINGMODE"],
+            sample_rate=exif["ZSAMPLERATE"],
+            track_format=exif["ZTRACKFORMAT"],
+            white_balance=exif["ZWHITEBALANCE"],
+            aperture=exif["ZAPERTURE"],
+            bit_rate=exif["ZBITRATE"],
+            duration=exif["ZDURATION"],
+            exposure_bias=exif["ZEXPOSUREBIAS"],
+            focal_length=exif["ZFOCALLENGTH"],
+            fps=exif["ZFPS"],
+            latitude=exif["ZLATITUDE"],
+            longitude=exif["ZLONGITUDE"],
+            shutter_speed=exif["ZSHUTTERSPEED"],
+            camera_make=exif["ZCAMERAMAKE"],
+            camera_model=exif["ZCAMERAMODEL"],
+            codec=exif["ZCODEC"],
+            lens_model=exif["ZLENSMODEL"],
+        )
+    except KeyError:
+        logging.debug(f"Could not find exif record for uuid {self.uuid}")
+        exif_info = ExifInfo(
+            iso=None,
+            flash_fired=None,
+            metering_mode=None,
+            sample_rate=None,
+            track_format=None,
+            white_balance=None,
+            aperture=None,
+            bit_rate=None,
+            duration=None,
+            exposure_bias=None,
+            focal_length=None,
+            fps=None,
+            latitude=None,
+            longitude=None,
+            shutter_speed=None,
+            camera_make=None,
+            camera_model=None,
+            codec=None,
+            lens_model=None,
+        )
+
+    return exif_info

osxphotos/photoinfo/_photoinfo_exiftool.py

@@ -0,0 +1,33 @@
+""" Implementation for PhotoInfo.exiftool property which returns ExifTool object for a photo """
+
+import logging
+import os
+
+from ..exiftool import ExifTool, get_exiftool_path
+
+@property
+def exiftool(self):
+    """ Returns an ExifTool object for the photo
+        requires that exiftool (https://exiftool.org/) be installed
+        If exiftool not installed, logs warning and returns None
+        If photo path is missing, returns None
+    """
+    try:
+        # return the memoized instance if it exists
+        return self._exiftool
+    except AttributeError:
+        try:
+            exiftool_path = get_exiftool_path()
+            if self.path is not None and os.path.isfile(self.path):
+                exiftool = ExifTool(self.path)
+            else:
+                exiftool = None
+                logging.debug(f"exiftool: missing path {self.uuid}")
+        except FileNotFoundError:
+            # get_exiftool_path raises FileNotFoundError if exiftool not found
+            exiftool = None
+            logging.warning(f"exiftool not in path; download and install from https://exiftool.org/")
+
+        self._exiftool = exiftool
+        return self._exiftool
+

osxphotos/photoinfo/_photoinfo_export.py

@@ -0,0 +1,999 @@
+""" export methods for PhotoInfo """
+
+# TODO: should this be its own PhotoExporter class?
+
+import filecmp
+import glob
+import json
+import logging
+import os
+import pathlib
+import re
+from collections import namedtuple  # pylint: disable=syntax-error
+
+from mako.template import Template
+
+from .._constants import (
+    _MAX_IPTC_KEYWORD_LEN,
+    _OSXPHOTOS_NONE_SENTINEL,
+    _TEMPLATE_DIR,
+    _UNKNOWN_PERSON,
+    _XMP_TEMPLATE_NAME,
+)
+from ..exiftool import ExifTool
+from .._export_db import ExportDBNoOp
+from .._filecmp import cmp_file, file_sig
+from ..utils import (
+    _copy_file,
+    _export_photo_uuid_applescript,
+    _hardlink_file,
+    dd_to_dms_str,
+)
+
+ExportResults = namedtuple(
+    "ExportResults", ["exported", "new", "updated", "skipped", "exif_updated"]
+)
+
+
+def export(
+    self,
+    dest,
+    *filename,
+    edited=False,
+    live_photo=False,
+    raw_photo=False,
+    export_as_hardlink=False,
+    overwrite=False,
+    increment=True,
+    sidecar_json=False,
+    sidecar_xmp=False,
+    use_photos_export=False,
+    timeout=120,
+    exiftool=False,
+    no_xattr=False,
+    use_albums_as_keywords=False,
+    use_persons_as_keywords=False,
+    keyword_template=None,
+):
+    """ export photo 
+        dest: must be valid destination path (or exception raised) 
+        filename: (optional): name of exported picture; if not provided, will use current filename 
+                    **NOTE**: if provided, user must ensure file extension (suffix) is correct. 
+                    For example, if photo is .CR2 file, edited image may be .jpeg.  
+                    If you provide an extension different than what the actual file is, 
+                    export will print a warning but will happily export the photo using the 
+                    incorrect file extension.  e.g. to get the extension of the edited photo, 
+                    reference PhotoInfo.path_edited
+        edited: (boolean, default=False); if True will export the edited version of the photo 
+                (or raise exception if no edited version) 
+        live_photo: (boolean, default=False); if True, will also export the associted .mov for live photos
+        raw_photo: (boolean, default=False); if True, will also export the associted RAW photo
+        export_as_hardlink: (boolean, default=False); if True, will hardlink files instead of copying them
+        overwrite: (boolean, default=False); if True will overwrite files if they alreay exist 
+        increment: (boolean, default=True); if True, will increment file name until a non-existant name is found 
+                    if overwrite=False and increment=False, export will fail if destination file already exists 
+        sidecar_json: (boolean, default = False); if True will also write a json sidecar with IPTC data in format readable by exiftool
+                    sidecar filename will be dest/filename.json 
+        sidecar_xmp: (boolean, default = False); if True will also write a XMP sidecar with IPTC data 
+                    sidecar filename will be dest/filename.xmp 
+        use_photos_export: (boolean, default=False); if True will attempt to export photo via applescript interaction with Photos
+        timeout: (int, default=120) timeout in seconds used with use_photos_export
+        exiftool: (boolean, default = False); if True, will use exiftool to write metadata to export file
+        no_xattr: (boolean, default = False); if True, exports file without preserving extended attributes
+        returns list of full paths to the exported files
+        use_albums_as_keywords: (boolean, default = False); if True, will include album names in keywords
+        when exporting metadata with exiftool or sidecar
+        use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords
+        when exporting metadata with exiftool or sidecar
+        keyword_template: (list of strings); list of template strings that will be rendered as used as keywords
+        returns: list of photos exported
+        """
+
+    # Implementation note: calls export2 to actually do the work
+
+    results = self.export2(
+        dest,
+        *filename,
+        edited=edited,
+        live_photo=live_photo,
+        raw_photo=raw_photo,
+        export_as_hardlink=export_as_hardlink,
+        overwrite=overwrite,
+        increment=increment,
+        sidecar_json=sidecar_json,
+        sidecar_xmp=sidecar_xmp,
+        use_photos_export=use_photos_export,
+        timeout=timeout,
+        exiftool=exiftool,
+        no_xattr=no_xattr,
+        use_albums_as_keywords=use_albums_as_keywords,
+        use_persons_as_keywords=use_persons_as_keywords,
+        keyword_template=keyword_template,
+    )
+
+    return results.exported
+
+
+def export2(
+    self,
+    dest,
+    *filename,
+    edited=False,
+    live_photo=False,
+    raw_photo=False,
+    export_as_hardlink=False,
+    overwrite=False,
+    increment=True,
+    sidecar_json=False,
+    sidecar_xmp=False,
+    use_photos_export=False,
+    timeout=120,
+    exiftool=False,
+    no_xattr=False,
+    use_albums_as_keywords=False,
+    use_persons_as_keywords=False,
+    keyword_template=None,
+    update=False,
+    export_db=None,
+):
+    """ export photo 
+        dest: must be valid destination path (or exception raised) 
+        filename: (optional): name of exported picture; if not provided, will use current filename 
+                    **NOTE**: if provided, user must ensure file extension (suffix) is correct. 
+                    For example, if photo is .CR2 file, edited image may be .jpeg.  
+                    If you provide an extension different than what the actual file is, 
+                    export will print a warning but will happily export the photo using the 
+                    incorrect file extension.  e.g. to get the extension of the edited photo, 
+                    reference PhotoInfo.path_edited
+        edited: (boolean, default=False); if True will export the edited version of the photo 
+                (or raise exception if no edited version) 
+        live_photo: (boolean, default=False); if True, will also export the associted .mov for live photos
+        raw_photo: (boolean, default=False); if True, will also export the associted RAW photo
+        export_as_hardlink: (boolean, default=False); if True, will hardlink files instead of copying them
+        overwrite: (boolean, default=False); if True will overwrite files if they alreay exist 
+        increment: (boolean, default=True); if True, will increment file name until a non-existant name is found 
+                    if overwrite=False and increment=False, export will fail if destination file already exists 
+        sidecar_json: (boolean, default = False); if True will also write a json sidecar with IPTC data in format readable by exiftool
+                    sidecar filename will be dest/filename.json 
+        sidecar_xmp: (boolean, default = False); if True will also write a XMP sidecar with IPTC data 
+                    sidecar filename will be dest/filename.xmp 
+        use_photos_export: (boolean, default=False); if True will attempt to export photo via applescript interaction with Photos
+        timeout: (int, default=120) timeout in seconds used with use_photos_export
+        exiftool: (boolean, default = False); if True, will use exiftool to write metadata to export file
+        no_xattr: (boolean, default = False); if True, exports file without preserving extended attributes
+        returns list of full paths to the exported files
+        use_albums_as_keywords: (boolean, default = False); if True, will include album names in keywords
+        when exporting metadata with exiftool or sidecar
+        use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords
+        when exporting metadata with exiftool or sidecar
+        keyword_template: (list of strings); list of template strings that will be rendered as used as keywords
+        update: (boolean, default=False); if True export will run in update mode, that is, it will
+                not export the photo if the current version already exists in the destination
+        export_db: (ExportDB_ABC); instance of a class that conforms to ExportDB_ABC with methods
+                for getting/setting data related to exported files to compare update state
+        Returns: ExportResults namedtuple with fields: exported, new, updated, skipped 
+                    where each field is a list of file paths
+            """
+
+    # if update, caller may pass function refs to get/set uuid for file being exported
+    # and for setting/getting the PhotoInfo json info for an exported file
+    if export_db is None:
+        export_db = ExportDBNoOp()
+
+    # suffix to add to edited files
+    # e.g. name will be filename_edited.jpg
+    edited_identifier = "_edited"
+
+    # list of all files exported during this call to export
+    exported_files = []
+
+    # list of new files during update
+    update_new_files = []
+
+    # list of files that were updated
+    update_updated_files = []
+
+    # list of all files skipped because they do not need to be updated (for use with update=True)
+    update_skipped_files = []
+
+    # check edited and raise exception trying to export edited version of
+    # photo that hasn't been edited
+    if edited and not self.hasadjustments:
+        raise ValueError(
+            "Photo does not have adjustments, cannot export edited version"
+        )
+
+    # check arguments and get destination path and filename (if provided)
+    if filename and len(filename) > 2:
+        raise TypeError(
+            "Too many positional arguments.  Should be at most two: destination, filename."
+        )
+    else:
+        # verify destination is a valid path
+        if dest is None:
+            raise ValueError("Destination must not be None")
+        elif not os.path.isdir(dest):
+            raise FileNotFoundError("Invalid path passed to export")
+
+        if filename and len(filename) == 1:
+            # if filename passed, use it
+            fname = filename[0]
+        else:
+            # no filename provided so use the default
+            # if edited file requested, use filename but add _edited
+            # need to use file extension from edited file as Photos saves a jpeg once edited
+            if edited and not use_photos_export:
+                # verify we have a valid path_edited and use that to get filename
+                if not self.path_edited:
+                    raise FileNotFoundError(
+                        "edited=True but path_edited is none; hasadjustments: "
+                        f" {self.hasadjustments}"
+                    )
+                edited_name = pathlib.Path(self.path_edited).name
+                edited_suffix = pathlib.Path(edited_name).suffix
+                fname = (
+                    pathlib.Path(self.filename).stem + edited_identifier + edited_suffix
+                )
+            else:
+                fname = self.filename
+
+    # check destination path
+    dest = pathlib.Path(dest)
+    fname = pathlib.Path(fname)
+    dest = dest / fname
+
+    # check extension of destination
+    if edited and self.path_edited is not None:
+        # use suffix from edited file
+        actual_suffix = pathlib.Path(self.path_edited).suffix
+    elif edited:
+        # use .jpeg as that's probably correct
+        # if edited and path_edited is None, will raise FileNotFoundError below
+        # unless use_photos_export is True
+        actual_suffix = ".jpeg"
+    else:
+        # use suffix from the non-edited file
+        actual_suffix = pathlib.Path(self.filename).suffix
+
+    # warn if suffixes don't match but ignore .JPG / .jpeg as
+    # Photo's often converts .JPG to .jpeg
+    suffixes = sorted([x.lower() for x in [dest.suffix, actual_suffix]])
+    if dest.suffix.lower() != actual_suffix.lower() and suffixes != [".jpeg", ".jpg"]:
+        logging.warning(
+            f"Invalid destination suffix: {dest.suffix}, should be {actual_suffix}"
+        )
+
+    # check to see if file exists and if so, add (1), (2), etc until we find one that works
+    # Photos checks the stem and adds (1), (2), etc which avoids collision with sidecars
+    # e.g. exporting sidecar for file1.png and file1.jpeg
+    # if file1.png exists and exporting file1.jpeg,
+    # dest will be file1 (1).jpeg even though file1.jpeg doesn't exist to prevent sidecar collision
+    if not update and increment and not overwrite:
+        count = 1
+        glob_str = str(dest.parent / f"{dest.stem}*")
+        dest_files = glob.glob(glob_str)
+        dest_files = [pathlib.Path(f).stem for f in dest_files]
+        dest_new = dest.stem
+        while dest_new in dest_files:
+            dest_new = f"{dest.stem} ({count})"
+            count += 1
+        dest = dest.parent / f"{dest_new}{dest.suffix}"
+
+    # TODO: need way to check if DB is missing, try to find the right photo anyway by seeing if they're the same and then updating
+    # move the checks into "if not use_photos_export" block below
+    # if use_photos_export is True then we'll export wether destination exists or not
+
+    # if overwrite==False and #increment==False, export should fail if file exists
+    if dest.exists() and not update and not overwrite and not increment:
+        raise FileExistsError(
+            f"destination exists ({dest}); overwrite={overwrite}, increment={increment}"
+        )
+
+    if not use_photos_export:
+        # find the source file on disk and export
+        # get path to source file and verify it's not None and is valid file
+        # TODO: how to handle ismissing or not hasadjustments and edited=True cases?
+        if edited:
+            if self.path_edited is not None:
+                src = self.path_edited
+            else:
+                raise FileNotFoundError(
+                    f"Cannot export edited photo if path_edited is None"
+                )
+        else:
+            if self.ismissing:
+                logging.debug(
+                    f"Attempting to export photo with ismissing=True: path = {self.path}"
+                )
+
+            if self.path is not None:
+                src = self.path
+            else:
+                raise FileNotFoundError("Cannot export photo if path is None")
+
+        if not os.path.isfile(src):
+            raise FileNotFoundError(f"{src} does not appear to exist")
+
+        logging.debug(
+            f"exporting {src} to {dest}, overwrite={overwrite}, increment={increment}, dest exists: {dest.exists()}"
+        )
+
+        # found source now try to find right destination
+        if update and dest.exists():
+            # destination exists, check to see if destination is the right UUID
+            dest_uuid = export_db.get_uuid_for_file(dest)
+            if dest_uuid is None and filecmp.cmp(src, dest):
+                # might be exporting into a pre-ExportDB folder or the DB got deleted
+                logging.debug(
+                    f"Found matching file with blank uuid: {self.uuid}, {dest}"
+                )
+                dest_uuid = self.uuid
+                export_db.set_uuid_for_file(dest, self.uuid)
+                export_db.set_info_for_uuid(self.uuid, self.json())
+                export_db.set_stat_orig_for_file(dest, file_sig(dest))
+                export_db.set_stat_exif_for_file(dest, (None, None, None))
+                export_db.set_exifdata_for_file(dest, None)
+            if dest_uuid != self.uuid:
+                # not the right file, find the right one
+                logging.debug(
+                    f"Need to find right photo: uuid={self.uuid}, dest={dest_uuid}, dest={dest}, path={self.path}"
+                )
+                count = 1
+                glob_str = str(dest.parent / f"{dest.stem} (*{dest.suffix}")
+                dest_files = glob.glob(glob_str)
+                found_match = False
+                for file_ in dest_files:
+                    dest_uuid = export_db.get_uuid_for_file(file_)
+                    if dest_uuid == self.uuid:
+                        logging.debug(
+                            f"Found matching file for uuid: {dest_uuid}, {file_}"
+                        )
+                        dest = pathlib.Path(file_)
+                        found_match = True
+                        break
+                    elif dest_uuid is None and filecmp.cmp(src, file_):
+                        # files match, update the UUID
+                        logging.debug(
+                            f"Found matching file with blank uuid: {self.uuid}, {file_}"
+                        )
+                        dest = pathlib.Path(file_)
+                        found_match = True
+                        export_db.set_uuid_for_file(file_, self.uuid)
+                        export_db.set_info_for_uuid(self.uuid, self.json())
+                        export_db.set_stat_orig_for_file(dest, file_sig(dest))
+                        export_db.set_stat_exif_for_file(dest, (None, None, None))
+                        export_db.set_exifdata_for_file(dest, None)
+                        break
+
+                if not found_match:
+                    logging.debug(
+                        f"Didn't find destination match for uuid {self.uuid} {dest}"
+                    )
+
+                    # increment the destination file
+                    count = 1
+                    glob_str = str(dest.parent / f"{dest.stem}*")
+                    dest_files = glob.glob(glob_str)
+                    dest_files = [pathlib.Path(f).stem for f in dest_files]
+                    dest_new = dest.stem
+                    while dest_new in dest_files:
+                        dest_new = f"{dest.stem} ({count})"
+                        count += 1
+                    dest = dest.parent / f"{dest_new}{dest.suffix}"
+                    logging.debug(f"New destination = {dest}, uuid = {self.uuid}")
+
+        # export the dest file
+        results = self._export_photo(
+            src,
+            dest,
+            update,
+            export_db,
+            overwrite,
+            no_xattr,
+            export_as_hardlink,
+            exiftool,
+        )
+        exported_files = results.exported
+        update_new_files = results.new
+        update_updated_files = results.updated
+        update_skipped_files = results.skipped
+
+        # copy live photo associated .mov if requested
+        if live_photo and self.live_photo:
+            live_name = dest.parent / f"{dest.stem}.mov"
+            src_live = self.path_live_photo
+
+            if src_live is not None:
+                logging.debug(
+                    f"Exporting live photo video of {filename} as {live_name.name}"
+                )
+                results = self._export_photo(
+                    src_live,
+                    live_name,
+                    update,
+                    export_db,
+                    overwrite,
+                    no_xattr,
+                    export_as_hardlink,
+                    exiftool,
+                )
+                exported_files.extend(results.exported)
+                update_new_files.extend(results.new)
+                update_updated_files.extend(results.updated)
+                update_skipped_files.extend(results.skipped)
+            else:
+                logging.debug(f"Skipping missing live movie for {filename}")
+
+        # copy associated RAW image if requested
+        if raw_photo and self.has_raw:
+            raw_path = pathlib.Path(self.path_raw)
+            raw_ext = raw_path.suffix
+            raw_name = dest.parent / f"{dest.stem}{raw_ext}"
+            if raw_path is not None:
+                logging.debug(f"Exporting RAW photo of {filename} as {raw_name.name}")
+                results = self._export_photo(
+                    raw_path,
+                    raw_name,
+                    update,
+                    export_db,
+                    overwrite,
+                    no_xattr,
+                    export_as_hardlink,
+                    exiftool,
+                )
+                exported_files.extend(results.exported)
+                update_new_files.extend(results.new)
+                update_updated_files.extend(results.updated)
+                update_skipped_files.extend(results.skipped)
+            else:
+                logging.debug(f"Skipping missing RAW photo for {filename}")
+    else:
+        # use_photo_export
+        exported = None
+        # export live_photo .mov file?
+        live_photo = True if live_photo and self.live_photo else False
+        if edited:
+            # exported edited version and not original
+            if filename:
+                # use filename stem provided
+                filestem = dest.stem
+            else:
+                # didn't get passed a filename, add _edited
+                filestem = f"{dest.stem}_edited"
+                dest = dest.parent / f"{filestem}.jpeg"
+
+            exported = _export_photo_uuid_applescript(
+                self.uuid,
+                dest.parent,
+                filestem=filestem,
+                original=False,
+                edited=True,
+                live_photo=live_photo,
+                timeout=timeout,
+                burst=self.burst,
+            )
+        else:
+            # export original version and not edited
+            filestem = dest.stem
+            exported = _export_photo_uuid_applescript(
+                self.uuid,
+                dest.parent,
+                filestem=filestem,
+                original=True,
+                edited=False,
+                live_photo=live_photo,
+                timeout=timeout,
+                burst=self.burst,
+            )
+
+        if exported is not None:
+            exported_files.extend(exported)
+        else:
+            logging.warning(
+                f"Error exporting photo {self.uuid} to {dest} with use_photos_export"
+            )
+
+    # export metadata
+    info = export_db.get_info_for_uuid(self.uuid)
+
+    if sidecar_json:
+        logging.debug("writing exiftool_json_sidecar")
+        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.json")
+        sidecar_str = self._exiftool_json_sidecar(
+            use_albums_as_keywords=use_albums_as_keywords,
+            use_persons_as_keywords=use_persons_as_keywords,
+            keyword_template=keyword_template,
+        )
+        try:
+            self._write_sidecar(sidecar_filename, sidecar_str)
+        except Exception as e:
+            logging.warning(f"Error writing json sidecar to {sidecar_filename}")
+            raise e
+
+    if sidecar_xmp:
+        logging.debug("writing xmp_sidecar")
+        sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.xmp")
+        sidecar_str = self._xmp_sidecar(
+            use_albums_as_keywords=use_albums_as_keywords,
+            use_persons_as_keywords=use_persons_as_keywords,
+            keyword_template=keyword_template,
+        )
+        try:
+            self._write_sidecar(sidecar_filename, sidecar_str)
+        except Exception as e:
+            logging.warning(f"Error writing xmp sidecar to {sidecar_filename}")
+            raise e
+
+    # if exiftool, write the metadata
+    if update:
+        exif_files = update_new_files + update_updated_files + update_skipped_files
+    else:
+        exif_files = exported_files
+
+    exif_files_updated = []
+    if exiftool and update and exif_files:
+        for exported_file in exif_files:
+            logging.debug(f"checking exif for {exported_file}")
+            files_are_different = False
+            old_data = export_db.get_exifdata_for_file(exported_file)
+            if old_data is not None:
+                old_data = json.loads(old_data)[0]
+                current_data = json.loads(
+                    self._exiftool_json_sidecar(
+                        use_albums_as_keywords=use_albums_as_keywords,
+                        use_persons_as_keywords=use_persons_as_keywords,
+                        keyword_template=keyword_template,
+                    )
+                )[0]
+                if old_data != current_data:
+                    files_are_different = True
+
+            if old_data is None or files_are_different:
+                # didn't have old data, assume we need to write it
+                # or files were different
+                logging.debug(f"No exifdata for {exported_file}, writing it")
+                self._write_exif_data(
+                    exported_file,
+                    use_albums_as_keywords=use_albums_as_keywords,
+                    use_persons_as_keywords=use_persons_as_keywords,
+                    keyword_template=keyword_template,
+                )
+                export_db.set_exifdata_for_file(
+                    exported_file,
+                    self._exiftool_json_sidecar(
+                        use_albums_as_keywords=use_albums_as_keywords,
+                        use_persons_as_keywords=use_persons_as_keywords,
+                        keyword_template=keyword_template,
+                    ),
+                )
+                export_db.set_stat_exif_for_file(exported_file, file_sig(exported_file))
+                exif_files_updated.append(exported_file)
+    elif exiftool and exif_files:
+        for exported_file in exif_files:
+            logging.debug(f"Writing exif data to {exported_file}")
+            self._write_exif_data(
+                exported_file,
+                use_albums_as_keywords=use_albums_as_keywords,
+                use_persons_as_keywords=use_persons_as_keywords,
+                keyword_template=keyword_template,
+            )
+            export_db.set_exifdata_for_file(
+                exported_file,
+                self._exiftool_json_sidecar(
+                    use_albums_as_keywords=use_albums_as_keywords,
+                    use_persons_as_keywords=use_persons_as_keywords,
+                    keyword_template=keyword_template,
+                ),
+            )
+            export_db.set_stat_exif_for_file(exported_file, file_sig(exported_file))
+            exif_files_updated.append(exported_file)
+
+    return ExportResults(
+        exported_files,
+        update_new_files,
+        update_updated_files,
+        update_skipped_files,
+        exif_files_updated,
+    )
+
+
+def _export_photo(
+    self,
+    src,
+    dest,
+    update,
+    export_db,
+    overwrite,
+    no_xattr,
+    export_as_hardlink,
+    exiftool,
+):
+    """ Helper function for export()
+        Does the actual copy or hardlink taking the appropriate 
+        action depending on update, overwrite
+        Assumes destination is the right destination (e.g. UUID matches)
+        sets UUID and JSON info foo exported file using set_uuid_for_file, set_inf_for_uuido
+        src: src path (string)
+        dest: dest path (pathlib.Path)
+        update: bool
+        export_db: instance of ExportDB that conforms to ExportDB_ABC interface
+        overwrite: bool
+        no_xattr: don't copy extended attributes
+        export_as_hardlink: bool
+        exiftool: bool
+        Returns: ExportResults
+    """
+
+    exported_files = []
+    update_updated_files = []
+    update_new_files = []
+    update_skipped_files = []
+
+    dest_str = str(dest)
+    dest_exists = dest.exists()
+    if export_as_hardlink:
+        # use hardlink instead of copy
+        if not update:
+            # not update, do the the hardlink
+            if overwrite and dest.exists():
+                # need to remove the destination first
+                dest.unlink()
+            logging.debug(f"Not update: export_as_hardlink linking file {src} {dest}")
+            _hardlink_file(src, dest)
+            export_db.set_uuid_for_file(dest_str, self.uuid)
+            export_db.set_info_for_uuid(self.uuid, self.json())
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            export_db.set_stat_exif_for_file(dest_str, (None, None, None))
+            export_db.set_exifdata_for_file(dest_str, None)
+            exported_files.append(dest_str)
+        elif dest_exists and dest.samefile(src):
+            # update, hardlink and it already points to the right file, do nothing
+            logging.debug(
+                f"Update: skipping samefile with export_as_hardlink {src} {dest}"
+            )
+            update_skipped_files.append(dest_str)
+        elif dest_exists:
+            # update, not the same file (e.g. user may not have used export_as_hardlink last time it was run
+            logging.debug(
+                f"Update: removing existing file prior to export_as_hardlink {src} {dest}"
+            )
+            dest.unlink()
+            _hardlink_file(src, dest)
+            export_db.set_uuid_for_file(dest_str, self.uuid)
+            export_db.set_info_for_uuid(self.uuid, self.json())
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            export_db.set_stat_exif_for_file(dest_str, (None, None, None))
+            export_db.set_exifdata_for_file(dest_str, None)
+            update_updated_files.append(dest_str)
+            exported_files.append(dest_str)
+        else:
+            # update, hardlink, destination doesn't exist (new file)
+            logging.debug(
+                f"Update: exporting new file with export_as_hardlink {src} {dest}"
+            )
+            _hardlink_file(src, dest)
+            export_db.set_uuid_for_file(dest_str, self.uuid)
+            export_db.set_info_for_uuid(self.uuid, self.json())
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            export_db.set_stat_exif_for_file(dest_str, (None, None, None))
+            export_db.set_exifdata_for_file(dest_str, None)
+            exported_files.append(dest_str)
+            update_new_files.append(dest_str)
+    else:
+        if not update:
+            # not update, do the the copy
+            if overwrite and dest.exists():
+                # need to remove the destination first
+                dest.unlink()
+            logging.debug(f"Not update: copying file {src} {dest}")
+            _copy_file(src, dest_str, norsrc=no_xattr)
+            export_db.set_uuid_for_file(dest_str, self.uuid)
+            export_db.set_info_for_uuid(self.uuid, self.json())
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            export_db.set_stat_exif_for_file(dest_str, (None, None, None))
+            export_db.set_exifdata_for_file(dest_str, None)
+            exported_files.append(dest_str)
+        # elif dest_exists and not exiftool and cmp_file(dest_str, export_db.get_stat_orig_for_file(dest_str)):
+        elif (
+            dest_exists
+            and not exiftool
+            and filecmp.cmp(src, dest)
+            and not dest.samefile(src)
+        ):
+            # destination exists but is identical
+            logging.debug(f"Update: skipping identifical original files {src} {dest}")
+            # call set_stat because code can reach this spot if no export DB but exporting a RAW or live photo
+            # potentially re-writes the data in the database but ensures database is complete
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            update_skipped_files.append(dest_str)
+        elif (
+            dest_exists
+            and exiftool
+            and cmp_file(dest_str, export_db.get_stat_exif_for_file(dest_str))
+            and not dest.samefile(src)
+        ):
+            # destination exists but is identical
+            logging.debug(f"Update: skipping identifical exiftool files {src} {dest}")
+            update_skipped_files.append(dest_str)
+        elif dest_exists:
+            # destination exists but is different or is a hardlink
+            logging.debug(f"Update: removing existing file prior to copy {src} {dest}")
+            stat_src = os.stat(src)
+            stat_dest = os.stat(dest)
+            dest.unlink()
+            _copy_file(src, dest_str, norsrc=no_xattr)
+            export_db.set_uuid_for_file(dest_str, self.uuid)
+            export_db.set_info_for_uuid(self.uuid, self.json())
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            export_db.set_stat_exif_for_file(dest_str, (None, None, None))
+            export_db.set_exifdata_for_file(dest_str, None)
+            exported_files.append(dest_str)
+            update_updated_files.append(dest_str)
+        else:
+            # destination doesn't exist, copy the file
+            logging.debug(f"Update: copying new file {src} {dest}")
+            _copy_file(src, dest_str, norsrc=no_xattr)
+            export_db.set_uuid_for_file(dest_str, self.uuid)
+            export_db.set_info_for_uuid(self.uuid, self.json())
+            export_db.set_stat_orig_for_file(dest_str, file_sig(dest_str))
+            export_db.set_stat_exif_for_file(dest_str, (None, None, None))
+            export_db.set_exifdata_for_file(dest_str, None)
+            exported_files.append(dest_str)
+            update_new_files.append(dest_str)
+
+    return ExportResults(
+        exported_files, update_new_files, update_updated_files, update_skipped_files, []
+    )
+
+
+def _write_exif_data(
+    self,
+    filepath,
+    use_albums_as_keywords=False,
+    use_persons_as_keywords=False,
+    keyword_template=None,
+):
+    """ write exif data to image file at filepath
+    filepath: full path to the image file """
+    if not os.path.exists(filepath):
+        raise FileNotFoundError(f"Could not find file {filepath}")
+    exiftool = ExifTool(filepath)
+    exif_info = json.loads(
+        self._exiftool_json_sidecar(
+            use_albums_as_keywords=use_albums_as_keywords,
+            use_persons_as_keywords=use_persons_as_keywords,
+            keyword_template=keyword_template,
+        )
+    )[0]
+    for exiftag, val in exif_info.items():
+        if type(val) == list:
+            # more than one, set first value the add additional values
+            exiftool.setvalue(exiftag, val.pop(0))
+            if val:
+                # add any remaining items
+                exiftool.addvalues(exiftag, *val)
+        else:
+            exiftool.setvalue(exiftag, val)
+
+
+def _exiftool_json_sidecar(
+    self,
+    use_albums_as_keywords=False,
+    use_persons_as_keywords=False,
+    keyword_template=None,
+):
+    """ return json string of EXIF details in exiftool sidecar format
+        Does not include all the EXIF fields as those are likely already in the image
+        use_albums_as_keywords: treat album names as keywords
+        use_persons_as_keywords: treat person names as keywords
+        keyword_template: (list of strings); list of template strings to render as keywords
+        Exports the following:
+            FileName
+            ImageDescription
+            Description
+            Title
+            TagsList
+            Keywords (may include album name, person name, or template)
+            Subject
+            PersonInImage
+            GPSLatitude, GPSLongitude
+            GPSPosition
+            GPSLatitudeRef, GPSLongitudeRef
+            DateTimeOriginal
+            OffsetTimeOriginal
+            ModifyDate """
+
+    exif = {}
+    exif["_CreatedBy"] = "osxphotos, https://github.com/RhetTbull/osxphotos"
+
+    if self.description:
+        exif["EXIF:ImageDescription"] = self.description
+        exif["XMP:Description"] = self.description
+
+    if self.title:
+        exif["XMP:Title"] = self.title
+
+    keyword_list = []
+    if self.keywords:
+        keyword_list.extend(self.keywords)
+
+    person_list = []
+    if self.persons:
+        # filter out _UNKNOWN_PERSON
+        person_list = sorted([p for p in self.persons if p != _UNKNOWN_PERSON])
+
+    if use_persons_as_keywords and person_list:
+        keyword_list.extend(sorted(person_list))
+
+    if use_albums_as_keywords and self.albums:
+        keyword_list.extend(sorted(self.albums))
+
+    if keyword_template:
+        rendered_keywords = []
+        for template_str in keyword_template:
+            rendered, unmatched = self.render_template(
+                template_str, none_str=_OSXPHOTOS_NONE_SENTINEL, path_sep="/"
+            )
+            if unmatched:
+                logging.warning(
+                    f"Unmatched template substitution for template: {template_str} {unmatched}"
+                )
+            rendered_keywords.extend(rendered)
+
+        # filter out any template values that didn't match by looking for sentinel
+        rendered_keywords = [
+            keyword
+            for keyword in sorted(rendered_keywords)
+            if _OSXPHOTOS_NONE_SENTINEL not in keyword
+        ]
+
+        # check to see if any keywords too long
+        long_keywords = [
+            long_str
+            for long_str in rendered_keywords
+            if len(long_str) > _MAX_IPTC_KEYWORD_LEN
+        ]
+        if long_keywords:
+            logging.warning(
+                f"Some keywords exceed max IPTC Keyword length of {_MAX_IPTC_KEYWORD_LEN}: {long_keywords}"
+            )
+
+        keyword_list.extend(rendered_keywords)
+
+    if keyword_list:
+        exif["XMP:TagsList"] = exif["IPTC:Keywords"] = keyword_list
+
+    if person_list:
+        exif["XMP:PersonInImage"] = person_list
+
+    if self.keywords or person_list:
+        # Photos puts both keywords and persons in Subject when using "Export IPTC as XMP"
+        # only use Photos' keywords for subject
+        exif["XMP:Subject"] = list(self.keywords) + person_list
+
+    # if self.favorite():
+    #     exif["Rating"] = 5
+
+    (lat, lon) = self.location
+    if lat is not None and lon is not None:
+        lat_str, lon_str = dd_to_dms_str(lat, lon)
+        exif["EXIF:GPSLatitude"] = lat_str
+        exif["EXIF:GPSLongitude"] = lon_str
+        exif["Composite:GPSPosition"] = f"{lat_str}, {lon_str}"
+        lat_ref = "North" if lat >= 0 else "South"
+        lon_ref = "East" if lon >= 0 else "West"
+        exif["EXIF:GPSLatitudeRef"] = lat_ref
+        exif["EXIF:GPSLongitudeRef"] = lon_ref
+
+    # process date/time and timezone offset
+    date = self.date
+    # exiftool expects format to "2015:01:18 12:00:00"
+    datetimeoriginal = date.strftime("%Y:%m:%d %H:%M:%S")
+    offsettime = date.strftime("%z")
+    # find timezone offset in format "-04:00"
+    offset = re.findall(r"([+-]?)([\d]{2})([\d]{2})", offsettime)
+    offset = offset[0]  # findall returns list of tuples
+    offsettime = f"{offset[0]}{offset[1]}:{offset[2]}"
+    exif["EXIF:DateTimeOriginal"] = datetimeoriginal
+    exif["EXIF:OffsetTimeOriginal"] = offsettime
+
+    if self.date_modified is not None:
+        exif["EXIF:ModifyDate"] = self.date_modified.strftime("%Y:%m:%d %H:%M:%S")
+
+    json_str = json.dumps([exif])
+    return json_str
+
+
+def _xmp_sidecar(
+    self,
+    use_albums_as_keywords=False,
+    use_persons_as_keywords=False,
+    keyword_template=None,
+):
+    """ returns string for XMP sidecar 
+        use_albums_as_keywords: treat album names as keywords
+        use_persons_as_keywords: treat person names as keywords
+        keyword_template: (list of strings); list of template strings to render as keywords """
+
+    # TODO: add additional fields to XMP file?
+
+    xmp_template = Template(filename=os.path.join(_TEMPLATE_DIR, _XMP_TEMPLATE_NAME))
+
+    keyword_list = []
+    if self.keywords:
+        keyword_list.extend(self.keywords)
+
+    # TODO: keyword handling in this and _exiftool_json_sidecar is
+    # good candidate for pulling out in a function
+
+    person_list = []
+    if self.persons:
+        # filter out _UNKNOWN_PERSON
+        person_list = [p for p in self.persons if p != _UNKNOWN_PERSON]
+
+    if use_persons_as_keywords and person_list:
+        keyword_list.extend(person_list)
+
+    if use_albums_as_keywords and self.albums:
+        keyword_list.extend(self.albums)
+
+    if keyword_template:
+        rendered_keywords = []
+        for template_str in keyword_template:
+            rendered, unmatched = self.render_template(
+                template_str, none_str=_OSXPHOTOS_NONE_SENTINEL, path_sep="/"
+            )
+            if unmatched:
+                logging.warning(
+                    f"Unmatched template substitution for template: {template_str} {unmatched}"
+                )
+            rendered_keywords.extend(rendered)
+
+        # filter out any template values that didn't match by looking for sentinel
+        rendered_keywords = [
+            keyword
+            for keyword in rendered_keywords
+            if _OSXPHOTOS_NONE_SENTINEL not in keyword
+        ]
+
+        # check to see if any keywords too long
+        long_keywords = [
+            long_str
+            for long_str in rendered_keywords
+            if len(long_str) > _MAX_IPTC_KEYWORD_LEN
+        ]
+        if long_keywords:
+            logging.warning(
+                f"Some keywords exceed max IPTC Keyword length of {_MAX_IPTC_KEYWORD_LEN}: {long_keywords}"
+            )
+
+        keyword_list.extend(rendered_keywords)
+
+    subject_list = []
+    if self.keywords or person_list:
+        # Photos puts both keywords and persons in Subject when using "Export IPTC as XMP"
+        subject_list = list(self.keywords) + person_list
+
+    xmp_str = xmp_template.render(
+        photo=self, keywords=keyword_list, persons=person_list, subjects=subject_list
+    )
+
+    # remove extra lines that mako inserts from template
+    xmp_str = "\n".join([line for line in xmp_str.split("\n") if line.strip() != ""])
+    return xmp_str
+
+
+def _write_sidecar(self, filename, sidecar_str):
+    """ write sidecar_str to filename
+        used for exporting sidecar info """
+    if not filename and not sidecar_str:
+        raise (
+            ValueError(
+                f"filename {filename} and sidecar_str {sidecar_str} must not be None"
+            )
+        )
+
+    # TODO: catch exception?
+    f = open(filename, "w")
+    f.write(sidecar_str)
+    f.close()

osxphotos/photoinfo/_photoinfo_searchinfo.py

@@ -0,0 +1,93 @@
+""" Methods and class for PhotoInfo exposing SearchInfo data such as labels 
+    Adds the following properties to PhotoInfo (valid only for Photos 5):
+        search_info: returns a SearchInfo object
+        labels: returns list of labels
+        labels_normalized: returns list of normalized labels
+"""
+
+from .._constants import _PHOTOS_4_VERSION, SEARCH_CATEGORY_LABEL
+
+
+@property
+def search_info(self):
+    """ returns SearchInfo object for photo 
+        only valid on Photos 5, on older libraries, returns None
+    """
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        return None
+
+    # memoize SearchInfo object
+    try:
+        return self._search_info
+    except AttributeError:
+        self._search_info = SearchInfo(self)
+        return self._search_info
+
+
+@property
+def labels(self):
+    """ returns list of labels applied to photo by Photos image categorization
+        only valid on Photos 5, on older libraries returns empty list 
+    """
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        return []
+
+    return self.search_info.labels
+
+
+@property
+def labels_normalized(self):
+    """ returns normalized list of labels applied to photo by Photos image categorization
+        only valid on Photos 5, on older libraries returns empty list 
+    """
+    if self._db._db_version <= _PHOTOS_4_VERSION:
+        return []
+
+    return self.search_info.labels_normalized
+
+
+class SearchInfo:
+    """ Info about search terms such as machine learning labels that Photos knows about a photo """
+
+    def __init__(self, photo):
+        """ photo: PhotoInfo object """
+
+        if photo._db._db_version <= _PHOTOS_4_VERSION:
+            raise NotImplementedError(
+                f"search info not implemented for this database version"
+            )
+
+        self._photo = photo
+        self.uuid = photo.uuid
+        try:
+            # get search info for this UUID
+            # there might not be any search info data (e.g. if Photo was missing or photoanalysisd not run yet)
+            self._db_searchinfo = photo._db._db_searchinfo_uuid[self.uuid]
+        except KeyError:
+            self._db_searchinfo = None
+
+    @property
+    def labels(self):
+        """ return list of labels associated with Photo """
+        if self._db_searchinfo:
+            labels = [
+                rec["content_string"]
+                for rec in self._db_searchinfo
+                if rec["category"] == SEARCH_CATEGORY_LABEL
+            ]
+        else:
+            labels = []
+        return labels
+
+    @property
+    def labels_normalized(self):
+        """ return list of normalized labels associated with Photo """
+        if self._db_searchinfo:
+            labels = [
+                rec["normalized_string"]
+                for rec in self._db_searchinfo
+                if rec["category"] == SEARCH_CATEGORY_LABEL
+            ]
+        else:
+            labels = []
+        return labels

osxphotos/photoinfo/photoinfo.py

@@ -4,9 +4,11 @@ Represents a single photo in the Photos library and provides access to the photo
 PhotosDB.photos() returns a list of PhotoInfo objects
 """
 
+import dataclasses
 import glob
 import json
 import logging
+import os
 import os.path
 import pathlib
 import re
@@ -16,26 +18,22 @@ from datetime import timedelta, timezone
 from pprint import pformat
 
 import yaml
-from mako.template import Template
 
-from ._constants import (
+from .._constants import (
     _MOVIE_TYPE,
     _PHOTO_TYPE,
     _PHOTOS_4_VERSION,
     _PHOTOS_5_SHARED_PHOTO_PATH,
-    _TEMPLATE_DIR,
-    _XMP_TEMPLATE_NAME,
+    _UNKNOWN_PERSON,
 )
-from .exiftool import ExifTool
-from .placeinfo import PlaceInfo4, PlaceInfo5
-from .albuminfo import AlbumInfo
-from .utils import (
-    _copy_file,
-    _export_photo_uuid_applescript,
-    _get_resource_loc,
-    dd_to_dms_str,
-    findfiles,
-    get_preferred_uti_extension,
+from ..albuminfo import AlbumInfo
+from ..datetime_formatter import DateTimeFormatter
+from ..placeinfo import PlaceInfo4, PlaceInfo5
+from ..utils import _debug, _get_resource_loc, findfiles, get_preferred_uti_extension
+from .template import (
+    MULTI_VALUE_SUBSTITUTIONS,
+    TEMPLATE_SUBSTITUTIONS,
+    TEMPLATE_SUBSTITUTIONS_MULTI_VALUED,
 )
 
 
@@ -45,6 +43,26 @@ class PhotoInfo:
     including keywords, persons, albums, uuid, path, etc.
     """
 
+    # import additional methods
+    from ._photoinfo_searchinfo import (
+        search_info,
+        labels,
+        labels_normalized,
+        SearchInfo,
+    )
+    from ._photoinfo_exifinfo import exif_info, ExifInfo
+    from ._photoinfo_exiftool import exiftool
+    from ._photoinfo_export import (
+        export,
+        export2,
+        _export_photo,
+        _exiftool_json_sidecar,
+        _write_exif_data,
+        _write_sidecar,
+        _xmp_sidecar,
+        ExportResults,
+    )
+
     def __init__(self, db=None, uuid=None, info=None):
         self._uuid = uuid
         self._info = info
@@ -53,7 +71,12 @@ class PhotoInfo:
     @property
     def filename(self):
         """ filename of the picture """
-        return self._info["filename"]
+        if self.has_raw and self.raw_original:
+            # return name of the RAW file
+            # TODO: not yet implemented
+            return self._info["filename"]
+        else:
+            return self._info["filename"]
 
     @property
     def original_filename(self):
@@ -237,7 +260,7 @@ class PhotoInfo:
             # if self._info["isMissing"] == 1:
             #     photopath = None  # path would be meaningless until downloaded
 
-            logging.debug(photopath)
+            # logging.debug(photopath)
 
         return photopath
 
@@ -296,9 +319,10 @@ class PhotoInfo:
             glob_str = f"{filestem}*.{raw_ext}"
             raw_file = findfiles(glob_str, filepath)
             if len(raw_file) != 1:
-                logging.warning(
-                    f"Error getting path to RAW file: {filepath}/{glob_str}"
-                )
+                # Note: In Photos Version 5.0 (141.19.150), images not copied to Photos Library
+                # that are missing do not always trigger is_missing = True as happens
+                # in earlier version so it's possible for this check to fail, if so, return None
+                logging.debug(f"Error getting path to RAW file: {filepath}/{glob_str}")
                 photopath = None
             else:
                 photopath = os.path.join(filepath, raw_file[0])
@@ -610,395 +634,341 @@ class PhotoInfo:
         """ returns True if associated RAW image and the RAW image is selected in Photos
             via "Use RAW as Original "
             otherwise returns False """
-        return True if self._info["original_resource_choice"] == 1 else False
+        return self._info["raw_is_original"]
 
-    def export(
-        self,
-        dest,
-        *filename,
-        edited=False,
-        live_photo=False,
-        raw_photo=False,
-        overwrite=False,
-        increment=True,
-        sidecar_json=False,
-        sidecar_xmp=False,
-        use_photos_export=False,
-        timeout=120,
-        exiftool=False,
-        no_xattr=False,
-    ):
-        """ export photo 
-            dest: must be valid destination path (or exception raised) 
-            filename: (optional): name of exported picture; if not provided, will use current filename 
-                      **NOTE**: if provided, user must ensure file extension (suffix) is correct. 
-                      For example, if photo is .CR2 file, edited image may be .jpeg.  
-                      If you provide an extension different than what the actual file is, 
-                      export will print a warning but will happily export the photo using the 
-                      incorrect file extension.  e.g. to get the extension of the edited photo, 
-                      reference PhotoInfo.path_edited
-            edited: (boolean, default=False); if True will export the edited version of the photo 
-                    (or raise exception if no edited version) 
-            live_photo: (boolean, default=False); if True, will also export the associted .mov for live photos
-            raw_photo: (boolean, default=False); if True, will also export the associted RAW photo
-            overwrite: (boolean, default=False); if True will overwrite files if they alreay exist 
-            increment: (boolean, default=True); if True, will increment file name until a non-existant name is found 
-                       if overwrite=False and increment=False, export will fail if destination file already exists 
-            sidecar_json: (boolean, default = False); if True will also write a json sidecar with IPTC data in format readable by exiftool
-                      sidecar filename will be dest/filename.json 
-            sidecar_xmp: (boolean, default = False); if True will also write a XMP sidecar with IPTC data 
-                      sidecar filename will be dest/filename.xmp 
-            use_photos_export: (boolean, default=False); if True will attempt to export photo via applescript interaction with Photos
-            timeout: (int, default=120) timeout in seconds used with use_photos_export
-            exiftool: (boolean, default = False); if True, will use exiftool to write metadata to export file
-            no_xattr: (boolean, default = False); if True, exports file without preserving extended attributes
-            returns list of full paths to the exported files """
+    def render_template(self, template, none_str="_", path_sep=None):
+        """ render a filename or directory template 
+            template: str template 
+            none_str: str to use default for None values, default is '_' 
+            path_sep: optional character to use as path separator, default is os.path.sep """
 
-        # list of all files exported during this call to export
-        exported_files = []
+        if path_sep is None:
+            path_sep = os.path.sep
+        elif path_sep is not None and len(path_sep) != 1:
+            raise ValueError(f"path_sep must be single character: {path_sep}")
 
-        # check edited and raise exception trying to export edited version of
-        # photo that hasn't been edited
-        if edited and not self.hasadjustments:
-            raise ValueError(
-                "Photo does not have adjustments, cannot export edited version"
-            )
+        # the rendering happens in two phases:
+        # phase 1: handle all the single-value template substitutions
+        #          results in a single string with all the template fields replaced
+        # phase 2: loop through all the multi-value template substitutions
+        #          could result in multiple strings
+        #          e.g. if template is "{album}/{person}" and there are 2 albums and 3 persons in the photo
+        #          there would be 6 possible renderings (2 albums x 3 persons)
 
-        # check arguments and get destination path and filename (if provided)
-        if filename and len(filename) > 2:
-            raise TypeError(
-                "Too many positional arguments.  Should be at most two: destination, filename."
-            )
-        else:
-            # verify destination is a valid path
-            if dest is None:
-                raise ValueError("Destination must not be None")
-            elif not os.path.isdir(dest):
-                raise FileNotFoundError("Invalid path passed to export")
+        # regex to find {template_field,optional_default} in strings
+        # for explanation of regex see https://regex101.com/r/4JJg42/1
+        # pylint: disable=anomalous-backslash-in-string
+        regex = r"(?<!\{)\{([^\\,}]+)(,{0,1}(([\w\-. ]+))?)(?=\}(?!\}))\}"
+        if type(template) is not str:
+            raise TypeError(f"template must be type str, not {type(template)}")
 
-            if filename and len(filename) == 1:
-                # if filename passed, use it
-                fname = filename[0]
-            else:
-                # no filename provided so use the default
-                # if edited file requested, use filename but add _edited
-                # need to use file extension from edited file as Photos saves a jpeg once edited
-                if edited and not use_photos_export:
-                    # verify we have a valid path_edited and use that to get filename
-                    if not self.path_edited:
-                        raise FileNotFoundError(
-                            "edited=True but path_edited is none; hasadjustments: "
-                            f" {self.hasadjustments}"
+        def make_subst_function(self, none_str, get_func=self.get_template_value):
+            """ returns: substitution function for use in re.sub 
+                none_str: value to use if substitution lookup is None and no default provided
+                get_func: function that gets the substitution value for a given template field
+                        default is get_template_value which handles the single-value fields """
+
+            # closure to capture photo, none_str in subst
+            def subst(matchobj):
+                groups = len(matchobj.groups())
+                if groups == 4:
+                    try:
+                        val = get_func(matchobj.group(1))
+                    except KeyError:
+                        return matchobj.group(0)
+
+                    if val is None:
+                        return (
+                            matchobj.group(3)
+                            if matchobj.group(3) is not None
+                            else none_str
                         )
-                    edited_name = pathlib.Path(self.path_edited).name
-                    edited_suffix = pathlib.Path(edited_name).suffix
-                    fname = pathlib.Path(self.filename).stem + "_edited" + edited_suffix
+                    else:
+                        return val
                 else:
-                    fname = self.filename
-
-        # check destination path
-        dest = pathlib.Path(dest)
-        fname = pathlib.Path(fname)
-        dest = dest / fname
-
-        # check extension of destination
-        if edited and self.path_edited is not None:
-            # use suffix from edited file
-            actual_suffix = pathlib.Path(self.path_edited).suffix
-        elif edited:
-            # use .jpeg as that's probably correct
-            # if edited and path_edited is None, will raise FileNotFoundError below
-            # unless use_photos_export is True
-            actual_suffix = ".jpeg"
-        else:
-            # use suffix from the non-edited file
-            actual_suffix = pathlib.Path(self.filename).suffix
-
-        # warn if suffixes don't match but ignore .JPG / .jpeg as
-        # Photo's often converts .JPG to .jpeg
-        suffixes = sorted([x.lower() for x in [dest.suffix, actual_suffix]])
-        if dest.suffix.lower() != actual_suffix.lower() and suffixes != [
-            ".jpeg",
-            ".jpg",
-        ]:
-            logging.warning(
-                f"Invalid destination suffix: {dest.suffix}, should be {actual_suffix}"
-            )
-
-        # check to see if file exists and if so, add (1), (2), etc until we find one that works
-        # Photos checks the stem and adds (1), (2), etc which avoids collision with sidecars
-        # e.g. exporting sidecar for file1.png and file1.jpeg
-        # if file1.png exists and exporting file1.jpeg,
-        # dest will be file1 (1).jpeg even though file1.jpeg doesn't exist to prevent sidecar collision
-        if increment and not overwrite:
-            count = 1
-            glob_str = str(dest.parent / f"{dest.stem}*")
-            dest_files = glob.glob(glob_str)
-            dest_files = [pathlib.Path(f).stem for f in dest_files]
-            dest_new = dest.stem
-            while dest_new in dest_files:
-                dest_new = f"{dest.stem} ({count})"
-                count += 1
-            dest = dest.parent / f"{dest_new}{dest.suffix}"
-
-        # if overwrite==False and #increment==False, export should fail if file exists
-        if dest.exists() and not overwrite and not increment:
-            raise FileExistsError(
-                f"destination exists ({dest}); overwrite={overwrite}, increment={increment}"
-            )
-
-        if not use_photos_export:
-            # find the source file on disk and export
-            # get path to source file and verify it's not None and is valid file
-            # TODO: how to handle ismissing or not hasadjustments and edited=True cases?
-            if edited:
-                if self.path_edited is not None:
-                    src = self.path_edited
-                else:
-                    raise FileNotFoundError(
-                        f"Cannot export edited photo if path_edited is None"
-                    )
-            else:
-                if self.ismissing:
-                    logging.warning(
-                        f"Attempting to export photo with ismissing=True: path = {self.path}"
+                    raise ValueError(
+                        f"Unexpected number of groups: expected 4, got {groups}"
                     )
 
-                if self.path is not None:
-                    src = self.path
-                else:
-                    raise FileNotFoundError("Cannot export photo if path is None")
+            return subst
 
-            if not os.path.isfile(src):
-                raise FileNotFoundError(f"{src} does not appear to exist")
+        subst_func = make_subst_function(self, none_str)
 
-            logging.debug(
-                f"exporting {src} to {dest}, overwrite={overwrite}, increment={increment}, dest exists: {dest.exists()}"
+        # do the replacements
+        rendered = re.sub(regex, subst_func, template)
+
+        # do multi-valued placements
+        # start with the single string from phase 1 above then loop through all
+        # multi-valued fields and all values for each of those fields
+        # rendered_strings will be updated as each field is processed
+        # for example: if two albums, two keywords, and one person and template is:
+        # "{created.year}/{album}/{keyword}/{person}"
+        # rendered strings would do the following:
+        # start (created.year filled in phase 1)
+        #   ['2011/{album}/{keyword}/{person}']
+        # after processing albums:
+        #   ['2011/Album1/{keyword}/{person}',
+        #    '2011/Album2/{keyword}/{person}',]
+        # after processing keywords:
+        #   ['2011/Album1/keyword1/{person}',
+        #    '2011/Album1/keyword2/{person}',
+        #    '2011/Album2/keyword1/{person}',
+        #    '2011/Album2/keyword2/{person}',]
+        # after processing person:
+        #   ['2011/Album1/keyword1/person1',
+        #    '2011/Album1/keyword2/person1',
+        #    '2011/Album2/keyword1/person1',
+        #    '2011/Album2/keyword2/person1',]
+
+        rendered_strings = set([rendered])
+        for field in MULTI_VALUE_SUBSTITUTIONS:
+            if field == "album":
+                values = self.albums
+            elif field == "keyword":
+                values = self.keywords
+            elif field == "person":
+                values = self.persons
+                # remove any _UNKNOWN_PERSON values
+                values = [val for val in values if val != _UNKNOWN_PERSON]
+            elif field == "label":
+                values = self.labels
+            elif field == "label_normalized":
+                values = self.labels_normalized
+            elif field == "folder_album":
+                values = []
+                # photos must be in an album to be in a folder
+                for album in self.album_info:
+                    if album.folder_names:
+                        # album in folder
+                        folder = path_sep.join(album.folder_names)
+                        folder += path_sep + album.title
+                        values.append(folder)
+                    else:
+                        # album not in folder
+                        values.append(album.title)
+            else:
+                raise ValueError(f"Unhandleded template value: {field}")
+
+            # If no values, insert None so code below will substite none_str for None
+            values = values or [None]
+
+            # Build a regex that matches only the field being processed
+            re_str = r"(?<!\\)\{(" + field + r")(,(([\w\-. ]{0,})))?\}"
+            regex_multi = re.compile(re_str)
+
+            # holds each of the new rendered_strings, set() to avoid duplicates
+            new_strings = set()
+
+            for str_template in rendered_strings:
+                for val in values:
+
+                    def get_template_value_multi(lookup_value):
+                        """ Closure passed to make_subst_function get_func 
+                            Capture val and field in the closure 
+                            Allows make_subst_function to be re-used w/o modification """
+                        if lookup_value == field:
+                            return val
+                        else:
+                            raise KeyError(f"Unexpected value: {lookup_value}")
+
+                    subst = make_subst_function(
+                        self, none_str, get_func=get_template_value_multi
+                    )
+                    new_string = regex_multi.sub(subst, str_template)
+                    new_strings.add(new_string)
+
+            # update rendered_strings for the next field to process
+            rendered_strings = new_strings
+
+        # find any {fields} that weren't replaced
+        unmatched = []
+        for rendered_str in rendered_strings:
+            unmatched.extend(
+                [
+                    no_match[0]
+                    for no_match in re.findall(regex, rendered_str)
+                    if no_match[0] not in unmatched
+                ]
             )
 
-            # copy the file, _copy_file uses ditto to preserve Mac extended attributes
-            _copy_file(src, dest, norsrc=no_xattr)
-            exported_files.append(str(dest))
+        # fix any escaped curly braces
+        rendered_strings = [
+            rendered_str.replace("{{", "{").replace("}}", "}")
+            for rendered_str in rendered_strings
+        ]
 
-            # copy live photo associated .mov if requested
-            if live_photo and self.live_photo:
-                live_name = dest.parent / f"{dest.stem}.mov"
-                src_live = self.path_live_photo
+        return rendered_strings, unmatched
 
-                if src_live is not None:
-                    logging.debug(
-                        f"Exporting live photo video of {filename} as {live_name.name}"
-                    )
-                    _copy_file(src_live, str(live_name), norsrc=no_xattr)
-                    exported_files.append(str(live_name))
-                else:
-                    logging.warning(f"Skipping missing live movie for {filename}")
+    def get_template_value(self, lookup):
+        """ lookup template value (single-value template substitutions) for use in make_subst_function
+            lookup: value to find a match for
+            returns: either the matching template value (which may be None)
+            raises: KeyError if no rule exists for lookup """
 
-            # copy associated RAW image if requested
-            if raw_photo and self.has_raw:
-                raw_path = pathlib.Path(self.path_raw)
-                raw_ext = raw_path.suffix
-                raw_name = dest.parent / f"{dest.stem}{raw_ext}"
-                if raw_path is not None:
-                    logging.debug(
-                        f"Exporting RAW photo of {filename} as {raw_name.name}"
-                    )
-                    _copy_file(str(raw_path), str(raw_name), norsrc=no_xattr)
-                    exported_files.append(str(raw_name))
-                else:
-                    logging.warning(f"Skipping missing RAW photo for {filename}")
-        else:
-            # use_photo_export
-            exported = None
-            # export live_photo .mov file?
-            live_photo = True if live_photo and self.live_photo else False
-            if edited:
-                # exported edited version and not original
-                if filename:
-                    # use filename stem provided
-                    filestem = dest.stem
-                else:
-                    # didn't get passed a filename, add _edited
-                    filestem = f"{dest.stem}_edited"
-                    dest = dest.parent / f"{filestem}.jpeg"
+        # must be a valid keyword
+        if lookup == "name":
+            return pathlib.Path(self.filename).stem
 
-                exported = _export_photo_uuid_applescript(
-                    self.uuid,
-                    dest.parent,
-                    filestem=filestem,
-                    original=False,
-                    edited=True,
-                    live_photo=live_photo,
-                    timeout=timeout,
-                    burst=self.burst,
-                )
-            else:
-                # export original version and not edited
-                filestem = dest.stem
-                exported = _export_photo_uuid_applescript(
-                    self.uuid,
-                    dest.parent,
-                    filestem=filestem,
-                    original=True,
-                    edited=False,
-                    live_photo=live_photo,
-                    timeout=timeout,
-                    burst=self.burst,
-                )
+        if lookup == "original_name":
+            return pathlib.Path(self.original_filename).stem
 
-            if exported is not None:
-                exported_files.extend(exported)
-            else:
-                logging.warning(
-                    f"Error exporting photo {self.uuid} to {dest} with use_photos_export"
-                )
+        if lookup == "title":
+            return self.title
 
-        if sidecar_json:
-            logging.debug("writing exiftool_json_sidecar")
-            sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.json")
-            sidecar_str = self._exiftool_json_sidecar()
-            try:
-                self._write_sidecar(sidecar_filename, sidecar_str)
-            except Exception as e:
-                logging.warning(f"Error writing json sidecar to {sidecar_filename}")
-                raise e
+        if lookup == "descr":
+            return self.description
 
-        if sidecar_xmp:
-            logging.debug("writing xmp_sidecar")
-            sidecar_filename = dest.parent / pathlib.Path(f"{dest.stem}.xmp")
-            sidecar_str = self._xmp_sidecar()
-            try:
-                self._write_sidecar(sidecar_filename, sidecar_str)
-            except Exception as e:
-                logging.warning(f"Error writing xmp sidecar to {sidecar_filename}")
-                raise e
+        if lookup == "created.date":
+            return DateTimeFormatter(self.date).date
 
-        # if exiftool, write the metadata
-        if exiftool and exported_files:
-            for exported_file in exported_files:
-                self._write_exif_data(exported_file)
+        if lookup == "created.year":
+            return DateTimeFormatter(self.date).year
 
-        return exported_files
+        if lookup == "created.yy":
+            return DateTimeFormatter(self.date).yy
 
-    def _write_exif_data(self, filepath):
-        """ write exif data to image file at filepath
-        filepath: full path to the image file """
-        if not os.path.exists(filepath):
-            raise FileNotFoundError(f"Could not find file {filepath}")
-        exiftool = ExifTool(filepath)
-        exif_info = json.loads(self._exiftool_json_sidecar())[0]
-        for exiftag, val in exif_info.items():
-            if type(val) == list:
-                # more than one, set first value the add additional values
-                exiftool.setvalue(exiftag, val.pop(0))
-                if val:
-                    # add any remaining items
-                    exiftool.addvalues(exiftag, *val)
-            else:
-                exiftool.setvalue(exiftag, val)
+        if lookup == "created.mm":
+            return DateTimeFormatter(self.date).mm
 
-    def _exiftool_json_sidecar(self):
-        """ return json string of EXIF details in exiftool sidecar format
-            Does not include all the EXIF fields as those are likely already in the image
-            Exports the following:
-                FileName
-                ImageDescription
-                Description
-                Title
-                TagsList
-                Keywords
-                Subject
-                PersonInImage
-                GPSLatitude, GPSLongitude
-                GPSPosition
-                GPSLatitudeRef, GPSLongitudeRef
-                DateTimeOriginal
-                OffsetTimeOriginal
-                ModifyDate """
+        if lookup == "created.month":
+            return DateTimeFormatter(self.date).month
 
-        exif = {}
-        exif["_CreatedBy"] = "osxphotos, https://github.com/RhetTbull/osxphotos"
+        if lookup == "created.mon":
+            return DateTimeFormatter(self.date).mon
 
-        if self.description:
-            exif["EXIF:ImageDescription"] = self.description
-            exif["XMP:Description"] = self.description
+        if lookup == "created.doy":
+            return DateTimeFormatter(self.date).doy
 
-        if self.title:
-            exif["XMP:Title"] = self.title
-
-        if self.keywords:
-            exif["XMP:TagsList"] = exif["IPTC:Keywords"] = list(self.keywords)
-            # Photos puts both keywords and persons in Subject when using "Export IPTC as XMP"
-            exif["XMP:Subject"] = list(self.keywords)
-
-        if self.persons:
-            exif["XMP:PersonInImage"] = self.persons
-            # Photos puts both keywords and persons in Subject when using "Export IPTC as XMP"
-            if "XMP:Subject" in exif:
-                exif["XMP:Subject"].extend(self.persons)
-            else:
-                exif["XMP:Subject"] = self.persons
-
-        # if self.favorite():
-        #     exif["Rating"] = 5
-
-        (lat, lon) = self.location
-        if lat is not None and lon is not None:
-            lat_str, lon_str = dd_to_dms_str(lat, lon)
-            exif["EXIF:GPSLatitude"] = lat_str
-            exif["EXIF:GPSLongitude"] = lon_str
-            exif["Composite:GPSPosition"] = f"{lat_str}, {lon_str}"
-            lat_ref = "North" if lat >= 0 else "South"
-            lon_ref = "East" if lon >= 0 else "West"
-            exif["EXIF:GPSLatitudeRef"] = lat_ref
-            exif["EXIF:GPSLongitudeRef"] = lon_ref
-
-        # process date/time and timezone offset
-        date = self.date
-        # exiftool expects format to "2015:01:18 12:00:00"
-        datetimeoriginal = date.strftime("%Y:%m:%d %H:%M:%S")
-        offsettime = date.strftime("%z")
-        # find timezone offset in format "-04:00"
-        offset = re.findall(r"([+-]?)([\d]{2})([\d]{2})", offsettime)
-        offset = offset[0]  # findall returns list of tuples
-        offsettime = f"{offset[0]}{offset[1]}:{offset[2]}"
-        exif["EXIF:DateTimeOriginal"] = datetimeoriginal
-        exif["EXIF:OffsetTimeOriginal"] = offsettime
-
-        if self.date_modified is not None:
-            exif["EXIF:ModifyDate"] = self.date_modified.strftime("%Y:%m:%d %H:%M:%S")
-
-        json_str = json.dumps([exif])
-        return json_str
-
-    def _xmp_sidecar(self):
-        """ returns string for XMP sidecar """
-        # TODO: add additional fields to XMP file?
-
-        xmp_template = Template(
-            filename=os.path.join(_TEMPLATE_DIR, _XMP_TEMPLATE_NAME)
-        )
-        xmp_str = xmp_template.render(photo=self)
-        # remove extra lines that mako inserts from template
-        xmp_str = "\n".join(
-            [line for line in xmp_str.split("\n") if line.strip() != ""]
-        )
-        return xmp_str
-
-    def _write_sidecar(self, filename, sidecar_str):
-        """ write sidecar_str to filename
-            used for exporting sidecar info """
-        if not filename and not sidecar_str:
-            raise (
-                ValueError(
-                    f"filename {filename} and sidecar_str {sidecar_str} must not be None"
-                )
+        if lookup == "modified.date":
+            return (
+                DateTimeFormatter(self.date_modified).date
+                if self.date_modified
+                else None
             )
 
-        # TODO: catch exception?
-        f = open(filename, "w")
-        f.write(sidecar_str)
-        f.close()
+        if lookup == "modified.year":
+            return (
+                DateTimeFormatter(self.date_modified).year
+                if self.date_modified
+                else None
+            )
+
+        if lookup == "modified.yy":
+            return (
+                DateTimeFormatter(self.date_modified).yy if self.date_modified else None
+            )
+
+        if lookup == "modified.mm":
+            return (
+                DateTimeFormatter(self.date_modified).mm if self.date_modified else None
+            )
+
+        if lookup == "modified.month":
+            return (
+                DateTimeFormatter(self.date_modified).month
+                if self.date_modified
+                else None
+            )
+
+        if lookup == "modified.mon":
+            return (
+                DateTimeFormatter(self.date_modified).mon
+                if self.date_modified
+                else None
+            )
+
+        if lookup == "modified.doy":
+            return (
+                DateTimeFormatter(self.date_modified).doy
+                if self.date_modified
+                else None
+            )
+
+        if lookup == "place.name":
+            return self.place.name if self.place else None
+
+        if lookup == "place.country_code":
+            return self.place.country_code if self.place else None
+
+        if lookup == "place.name.country":
+            return (
+                self.place.names.country[0]
+                if self.place and self.place.names.country
+                else None
+            )
+
+        if lookup == "place.name.state_province":
+            return (
+                self.place.names.state_province[0]
+                if self.place and self.place.names.state_province
+                else None
+            )
+
+        if lookup == "place.name.city":
+            return (
+                self.place.names.city[0]
+                if self.place and self.place.names.city
+                else None
+            )
+
+        if lookup == "place.name.area_of_interest":
+            return (
+                self.place.names.area_of_interest[0]
+                if self.place and self.place.names.area_of_interest
+                else None
+            )
+
+        if lookup == "place.address":
+            return (
+                self.place.address_str
+                if self.place and self.place.address_str
+                else None
+            )
+
+        if lookup == "place.address.street":
+            return (
+                self.place.address.street
+                if self.place and self.place.address.street
+                else None
+            )
+
+        if lookup == "place.address.city":
+            return (
+                self.place.address.city
+                if self.place and self.place.address.city
+                else None
+            )
+
+        if lookup == "place.address.state_province":
+            return (
+                self.place.address.state_province
+                if self.place and self.place.address.state_province
+                else None
+            )
+
+        if lookup == "place.address.postal_code":
+            return (
+                self.place.address.postal_code
+                if self.place and self.place.address.postal_code
+                else None
+            )
+
+        if lookup == "place.address.country":
+            return (
+                self.place.address.country
+                if self.place and self.place.address.country
+                else None
+            )
+
+        if lookup == "place.address.country_code":
+            return (
+                self.place.address.iso_country_code
+                if self.place and self.place.address.iso_country_code
+                else None
+            )
+
+        # if here, didn't get a match
+        raise KeyError(f"No rule for processing {lookup}")
 
     @property
     def _longitude(self):
@@ -1069,6 +1039,9 @@ class PhotoInfo:
         date_modified_iso = (
             self.date_modified.isoformat() if self.date_modified else None
         )
+        folders = {album.title: album.folder_names for album in self.album_info}
+        exif = dataclasses.asdict(self.exif_info) if self.exif_info else {}
+        place = self.place.as_dict() if self.place else {}
 
         pic = {
             "uuid": self.uuid,
@@ -1078,7 +1051,10 @@ class PhotoInfo:
             "description": self.description,
             "title": self.title,
             "keywords": self.keywords,
+            "labels": self.labels,
+            "keywords": self.keywords,
             "albums": self.albums,
+            "folders": folders,
             "persons": self.persons,
             "path": self.path,
             "ismissing": self.ismissing,
@@ -1109,6 +1085,8 @@ class PhotoInfo:
             "has_raw": self.has_raw,
             "uti_raw": self.uti_raw,
             "path_raw": self.path_raw,
+            "place": place,
+            "exif": exif,
         }
         return json.dumps(pic)
 

osxphotos/photoinfo/template.py

@@ -0,0 +1,66 @@
+""" Custom template system for osxphotos (implemented in PhotoInfo.render_template) """
+
+# Rolled my own template system because:
+# 1. Needed to handle multiple values (e.g. album, keyword)
+# 2. Needed to handle default values if template not found
+# 3. Didn't want user to need to know python (e.g. by using Mako which is
+#    already used elsewhere in this project)
+# 4. Couldn't figure out how to do #1 and #2 with str.format()
+#
+# This code isn't elegant but it seems to work well.  PRs gladly accepted.
+
+import locale
+
+# ensure locale set to user's locale
+locale.setlocale(locale.LC_ALL, "")
+
+# Permitted substitutions (each of these returns a single value or None)
+TEMPLATE_SUBSTITUTIONS = {
+    "{name}": "Current filename of the photo",
+    "{original_name}": "Photo's original filename when imported to Photos",
+    "{title}": "Title of the photo",
+    "{descr}": "Description of the photo",
+    "{created.date}": "Photo's creation date in ISO format, e.g. '2020-03-22'",
+    "{created.year}": "4-digit year of file creation time",
+    "{created.yy}": "2-digit year of file creation time",
+    "{created.mm}": "2-digit month of the file creation time (zero padded)",
+    "{created.month}": "Month name in user's locale of the file creation time",
+    "{created.mon}": "Month abbreviation in the user's locale of the file creation time",
+    "{created.doy}": "3-digit day of year (e.g Julian day) of file creation time, starting from 1 (zero padded)",
+    "{modified.date}": "Photo's modification date in ISO format, e.g. '2020-03-22'",
+    "{modified.year}": "4-digit year of file modification time",
+    "{modified.yy}": "2-digit year of file modification time",
+    "{modified.mm}": "2-digit month of the file modification time (zero padded)",
+    "{modified.month}": "Month name in user's locale of the file modification time",
+    "{modified.mon}": "Month abbreviation in the user's locale of the file modification time",
+    "{modified.doy}": "3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)",
+    "{place.name}": "Place name from the photo's reverse geolocation data, as displayed in Photos",
+    "{place.country_code}": "The ISO country code from the photo's reverse geolocation data",
+    "{place.name.country}": "Country name from the photo's reverse geolocation data",
+    "{place.name.state_province}": "State or province name from the photo's reverse geolocation data",
+    "{place.name.city}": "City or locality name from the photo's reverse geolocation data",
+    "{place.name.area_of_interest}": "Area of interest name (e.g. landmark or public place) from the photo's reverse geolocation data",
+    "{place.address}": "Postal address from the photo's reverse geolocation data, e.g. '2007 18th St NW, Washington, DC 20009, United States'",
+    "{place.address.street}": "Street part of the postal address, e.g. '2007 18th St NW'",
+    "{place.address.city}": "City part of the postal address, e.g. 'Washington'",
+    "{place.address.state_province}": "State/province part of the postal address, e.g. 'DC'",
+    "{place.address.postal_code}": "Postal code part of the postal address, e.g. '20009'",
+    "{place.address.country}": "Country name of the postal address, e.g. 'United States'",
+    "{place.address.country_code}": "ISO country code of the postal address, e.g. 'US'",
+}
+
+# Permitted multi-value substitutions (each of these returns None or 1 or more values)
+TEMPLATE_SUBSTITUTIONS_MULTI_VALUED = {
+    "{album}": "Album(s) photo is contained in",
+    "{folder_album}": "Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder",
+    "{keyword}": "Keyword(s) assigned to photo",
+    "{person}": "Person(s) / face(s) in a photo",
+    "{label}": "Image categorization label associated with a photo (Photos 5 only)",
+    "{label_normalized}": "All lower case version of 'label' (Photos 5 only)",
+}
+
+# Just the multi-valued substitution names without the braces
+MULTI_VALUE_SUBSTITUTIONS = [
+    field.replace("{", "").replace("}", "")
+    for field in TEMPLATE_SUBSTITUTIONS_MULTI_VALUED.keys()
+]

osxphotos/photosdb/__init__.py

@@ -0,0 +1,6 @@
+"""
+PhotosDB class
+Processes a Photos.app library database to extract information about photos
+"""
+
+from .photosdb import PhotosDB

osxphotos/photosdb/_photosdb_process_exif.py

@@ -0,0 +1,56 @@
+""" PhotosDB method for processing exif info 
+    Do not import this module directly """
+
+import logging
+
+from .._constants import _PHOTOS_4_VERSION
+from ..utils import _db_is_locked, _debug, _open_sql_file
+
+
+def _process_exifinfo(self):
+    """ load the exif data from the database 
+        this is a PhotosDB method that should be imported in 
+        the PhotosDB class definition in photosdb.py
+    """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        _process_exifinfo_4(self)
+    else:
+        _process_exifinfo_5(self)
+
+
+# The following methods do not get imported into PhotosDB
+# but will get called by _process_exifinfo
+
+
+def _process_exifinfo_4(photosdb):
+    """ process exif info for Photos <= 4 
+        photosdb: PhotosDB instance """
+    photosdb._db_exifinfo_uuid = {}
+    raise NotImplementedError(f"search info not implemented for this database version")
+
+
+def _process_exifinfo_5(photosdb):
+    """ process exif info for Photos >= 5 
+        photosdb: PhotosDB instance """
+
+    db = photosdb._tmp_db
+    
+    (conn, cursor) = _open_sql_file(db)
+
+    result = conn.execute(
+        """
+        SELECT ZGENERICASSET.ZUUID, ZEXTENDEDATTRIBUTES.*
+        FROM ZGENERICASSET
+        JOIN ZEXTENDEDATTRIBUTES
+        ON ZEXTENDEDATTRIBUTES.ZASSET = ZGENERICASSET.Z_PK
+        """
+    )
+
+    photosdb._db_exifinfo_uuid = {}
+    cols = [c[0] for c in result.description]
+    for row in result.fetchall():
+        record = dict(zip(cols, row))
+        uuid = record["ZUUID"]
+        if uuid in photosdb._db_exifinfo_uuid:
+            logging.warning(f"duplicate exifinfo record found for uuid {uuid}")
+        photosdb._db_exifinfo_uuid[uuid] = record

osxphotos/photosdb/_photosdb_process_searchinfo.py

@@ -0,0 +1,197 @@
+""" Methods for PhotosDB to add Photos 5 search info such as machine learning labels 
+    Kudos to Simon Willison who figured out how to extract this data from psi.sql
+    ref: https://github.com/dogsheep/photos-to-sqlite/issues/16
+"""
+
+import logging
+import pathlib
+import uuid as uuidlib
+from pprint import pformat
+
+from .._constants import _PHOTOS_4_VERSION, SEARCH_CATEGORY_LABEL
+from ..utils import _db_is_locked, _debug, _open_sql_file
+
+"""
+    This module should be imported in the class defintion of PhotosDB in photosdb.py
+    Do not import this module directly
+    This module adds the following method to PhotosDB:
+        _process_searchinfo: process search terms from psi.sqlite 
+
+    The following properties are added to PhotosDB    
+        labels: list of all labels in the library
+        labels_normalized: list of all labels normalized in the library  
+        labels_as_dict: dict of {label: count of photos} in reverse sorted order (most photos first)
+        labels_normalized_as_dict: dict of {normalized label: count of photos} in reverse sorted order (most photos first)
+
+    The following data structures are added to PhotosDB
+        self._db_searchinfo_categories
+        self._db_searchinfo_uuid
+        self._db_searchinfo_labels 
+        self._db_searchinfo_labels_normalized 
+    
+    These methods only work on Photos 5 databases.  Will print warning on earlier library versions.
+"""
+
+
+def _process_searchinfo(self):
+    """ load machine learning/search term label info from a Photos library
+        db_connection: a connection to the SQLite database file containing the 
+        search terms.  In Photos 5, this is called psi.sqlite
+        Note: Only works on Photos version == 5.0 """
+
+    if self._db_version <= _PHOTOS_4_VERSION:
+        raise NotImplementedError(
+            f"search info not implemented for this database version"
+        )
+
+    search_db_path = pathlib.Path(self._dbfile).parent / "search" / "psi.sqlite"
+    if not search_db_path.exists():
+        raise FileNotFoundError(f"could not find search db: {search_db_path}")
+
+    if _db_is_locked(search_db_path):
+        search_db = self._copy_db_file(search_db_path)
+    else:
+        search_db = search_db_path
+
+    (conn, c) = _open_sql_file(search_db)
+
+    result = conn.execute(
+        """
+        select
+        ga.rowid,
+        assets.uuid_0,
+        assets.uuid_1,
+        groups.rowid as groupid,
+        groups.category,
+        groups.owning_groupid,
+        groups.content_string,
+        groups.normalized_string,
+        groups.lookup_identifier
+        from
+        ga
+        join groups on groups.rowid = ga.groupid
+        join assets on ga.assetid = assets.rowid
+        order by
+        ga.rowid
+        """
+    )
+
+    # _db_searchinfo_uuid is dict in form {uuid : [list of associated search info records]
+    _db_searchinfo_uuid = {}
+
+    # _db_searchinfo_categories is dict in form {search info category id: list normalized strings for the category
+    # right now, this is mostly for debugging to easily see which search terms are in the library
+    _db_searchinfo_categories = {}
+
+    # _db_searchinfo_labels is dict in form {normalized label: [list of photo uuids]}
+    # this serves as a reverse index from label to photos containing the label
+    # _db_searchinfo_labels_normalized is the same but with normalized (lower case) version of the label
+    _db_searchinfo_labels = {}
+    _db_searchinfo_labels_normalized = {}
+
+    cols = [c[0] for c in result.description]
+    for row in result.fetchall():
+        record = dict(zip(cols, row))
+        uuid = ints_to_uuid(record["uuid_0"], record["uuid_1"])
+        # strings have null character appended, so strip it
+        for key in record:
+            if isinstance(record[key], str):
+                record[key] = record[key].replace("\x00", "")
+        try:
+            _db_searchinfo_uuid[uuid].append(record)
+        except KeyError:
+            _db_searchinfo_uuid[uuid] = [record]
+
+        category = record["category"]
+        try:
+            _db_searchinfo_categories[record["category"]].append(
+                record["normalized_string"]
+            )
+        except KeyError:
+            _db_searchinfo_categories[record["category"]] = [
+                record["normalized_string"]
+            ]
+
+        if record["category"] == SEARCH_CATEGORY_LABEL:
+            label = record["content_string"]
+            label_norm = record["normalized_string"]
+            try:
+                _db_searchinfo_labels[label].append(uuid)
+                _db_searchinfo_labels_normalized[label_norm].append(uuid)
+            except KeyError:
+                _db_searchinfo_labels[label] = [uuid]
+                _db_searchinfo_labels_normalized[label_norm] = [uuid]
+
+    self._db_searchinfo_categories = _db_searchinfo_categories
+    self._db_searchinfo_uuid = _db_searchinfo_uuid
+    self._db_searchinfo_labels = _db_searchinfo_labels
+    self._db_searchinfo_labels_normalized = _db_searchinfo_labels_normalized
+
+    if _debug():
+        logging.debug(
+            "_db_searchinfo_categories: \n" + pformat(self._db_searchinfo_categories)
+        )
+        logging.debug("_db_searchinfo_uuid: \n" + pformat(self._db_searchinfo_uuid))
+        logging.debug("_db_searchinfo_labels: \n" + pformat(self._db_searchinfo_labels))
+        logging.debug(
+            "_db_searchinfo_labels_normalized: \n"
+            + pformat(self._db_searchinfo_labels_normalized)
+        )
+
+
+@property
+def labels(self):
+    """ return list of all search info labels found in the library """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return []
+
+    return list(self._db_searchinfo_labels.keys())
+
+
+@property
+def labels_normalized(self):
+    """ return list of all normalized search info labels found in the library """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return []
+
+    return list(self._db_searchinfo_labels_normalized.keys())
+
+
+@property
+def labels_as_dict(self):
+    """ return labels as dict of label: count in reverse sorted order (descending) """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return dict()
+
+    labels = {k: len(v) for k, v in self._db_searchinfo_labels.items()}
+    labels = dict(sorted(labels.items(), key=lambda kv: kv[1], reverse=True))
+    return labels
+
+
+@property
+def labels_normalized_as_dict(self):
+    """ return normalized labels as dict of label: count in reverse sorted order (descending) """
+    if self._db_version <= _PHOTOS_4_VERSION:
+        logging.warning(f"SearchInfo not implemented for this library version")
+        return dict()
+    labels = {k: len(v) for k, v in self._db_searchinfo_labels_normalized.items()}
+    labels = dict(sorted(labels.items(), key=lambda kv: kv[1], reverse=True))
+    return labels
+
+
+# The following method is not imported into PhotosDB
+
+
+def ints_to_uuid(uuid_0, uuid_1):
+    """ convert two signed ints into a UUID strings
+        uuid_0, uuid_1: the two int components of an RFC 4122 UUID """
+
+    # assumes uuid imported as uuidlib (to avoid namespace conflict with other uses of uuid)
+
+    bytes_ = uuid_0.to_bytes(8, "little", signed=True) + uuid_1.to_bytes(
+        8, "little", signed=True
+    )
+    return str(uuidlib.UUID(bytes=bytes_)).upper()

osxphotos/photosdb/photosdb.py

@@ -15,26 +15,27 @@ from datetime import datetime
 from pprint import pformat
 from shutil import copyfile
 
-from ._constants import (
+from .._constants import (
     _MOVIE_TYPE,
     _PHOTO_TYPE,
     _PHOTOS_3_VERSION,
+    _PHOTOS_4_ALBUM_KIND,
+    _PHOTOS_4_ROOT_FOLDER,
+    _PHOTOS_4_TOP_LEVEL_ALBUM,
     _PHOTOS_4_VERSION,
+    _PHOTOS_5_ALBUM_KIND,
+    _PHOTOS_5_FOLDER_KIND,
+    _PHOTOS_5_ROOT_FOLDER_KIND,
+    _PHOTOS_5_SHARED_ALBUM_KIND,
     _PHOTOS_5_VERSION,
     _TESTED_DB_VERSIONS,
     _TESTED_OS_VERSIONS,
     _UNKNOWN_PERSON,
-    _PHOTOS_4_ALBUM_KIND,
-    _PHOTOS_4_TOP_LEVEL_ALBUM,
-    _PHOTOS_5_ROOT_FOLDER_KIND,
-    _PHOTOS_5_FOLDER_KIND,
-    _PHOTOS_5_ALBUM_KIND,
-    _PHOTOS_5_SHARED_ALBUM_KIND,
 )
-from ._version import __version__
-from .albuminfo import AlbumInfo, FolderInfo
-from .photoinfo import PhotoInfo
-from .utils import (
+from .._version import __version__
+from ..albuminfo import AlbumInfo, FolderInfo
+from ..photoinfo import PhotoInfo
+from ..utils import (
     _check_file_exists,
     _db_is_locked,
     _debug,
@@ -43,17 +44,27 @@ from .utils import (
     get_last_library_path,
 )
 
+
 # TODO: Add test for imageTimeZoneOffsetSeconds = None
 # TODO: Fix command line so multiple --keyword, etc. are AND (instead of OR as they are in .photos())
 #       Or fix the help text to match behavior
 # TODO: Add test for __str__
 # TODO: Add special albums and magic albums
-# TODO: fix "if X not in y" dictionary checks to use try/except EAFP style
 
 
 class PhotosDB:
     """ Processes a Photos.app library database to extract information about photos """
 
+    # import additional methods
+    from ._photosdb_process_exif import _process_exifinfo
+    from ._photosdb_process_searchinfo import (
+        _process_searchinfo,
+        labels,
+        labels_normalized,
+        labels_as_dict,
+        labels_normalized_as_dict,
+    )
+
     def __init__(self, *dbfile_, dbfile=None):
         """ create a new PhotosDB object 
             path to photos library or database may be specified EITHER as first argument or as named argument dbfile=path 
@@ -77,6 +88,12 @@ class PhotosDB:
 
         # set up the data structures used to store all the Photo database info
 
+        # if True, will treat persons as keywords when exporting metadata
+        self.use_persons_as_keywords = False
+
+        # if True, will treat albums as keywords when exporting metadata
+        self.use_albums_as_keywords = False
+
         # Path to the Photos library database file
         # photos.db in the photos library database/ directory
         self._dbfile = None
@@ -240,7 +257,7 @@ class PhotosDB:
         self._db_version = self._get_db_version()
 
         # If Photos >= 5, actual data isn't in photos.db but in Photos.sqlite
-        if int(self._db_version) >= int(_PHOTOS_5_VERSION):
+        if int(self._db_version) > int(_PHOTOS_4_VERSION):
             dbpath = pathlib.Path(self._dbfile).parent
             dbfile = dbpath / "Photos.sqlite"
             if not _check_file_exists(dbfile):
@@ -259,7 +276,7 @@ class PhotosDB:
         library_path = os.path.dirname(os.path.abspath(dbfile))
         (library_path, _) = os.path.split(library_path)  # drop /database from path
         self._library_path = library_path
-        if int(self._db_version) < int(_PHOTOS_5_VERSION):
+        if int(self._db_version) <= int(_PHOTOS_4_VERSION):
             masters_path = os.path.join(library_path, "Masters")
             self._masters_path = masters_path
         else:
@@ -528,7 +545,6 @@ class PhotosDB:
         """ process the Photos database to extract info
             works on Photos version <= 4.0 """
 
-        # TODO: Update strings to remove + (not needed)
         # Epoch is Jan 1, 2001
         td = (datetime(2001, 1, 1, 0, 0) - datetime(1970, 1, 1, 0, 0)).total_seconds()
 
@@ -656,8 +672,11 @@ class PhotosDB:
         # build folder hierarchy
         for album, details in self._dbalbum_details.items():
             parent_folder = details["folderUuid"]
-            if parent_folder != _PHOTOS_4_TOP_LEVEL_ALBUM:
-                # logging.warning(f"album = {details['title']}, parent = {parent_folder}")
+            if details[
+                "albumSubclass"
+            ] == _PHOTOS_4_ALBUM_KIND and parent_folder not in [
+                _PHOTOS_4_TOP_LEVEL_ALBUM
+            ]:
                 folder_hierarchy = self._build_album_folder_hierarchy_4(parent_folder)
                 self._dbalbum_folders[album] = folder_hierarchy
             else:
@@ -779,14 +798,20 @@ class PhotosDB:
             # There are sometimes negative values for lastmodifieddate in the database
             # I don't know what these mean but they will raise exception in datetime if
             # not accounted for
-            if row[4] is not None and row[4] >= 0:
+            try:
                 self._dbphotos[uuid]["lastmodifieddate"] = datetime.fromtimestamp(
                     row[4] + td
                 )
-            else:
+            except ValueError:
+                self._dbphotos[uuid]["lastmodifieddate"] = None
+            except TypeError:
                 self._dbphotos[uuid]["lastmodifieddate"] = None
 
-            self._dbphotos[uuid]["imageDate"] = datetime.fromtimestamp(row[5] + td)
+            try:
+                self._dbphotos[uuid]["imageDate"] = datetime.fromtimestamp(row[5] + td)
+            except ValueError:
+                self._dbphotos[uuid]["imageDate"] = datetime.date(1970, 1, 1)
+
             self._dbphotos[uuid]["mainRating"] = row[6]
             self._dbphotos[uuid]["hasAdjustments"] = row[7]
             self._dbphotos[uuid]["hasKeywords"] = row[8]
@@ -888,6 +913,7 @@ class PhotosDB:
             # TODO: NOT YET USED -- PLACEHOLDER for RAW processing (currently only in _process_database5)
             # original resource choice (e.g. RAW or jpeg)
             self._dbphotos[uuid]["original_resource_choice"] = None
+            self._dbphotos[uuid]["raw_is_original"] = None
 
             # associated RAW image info
             self._dbphotos[uuid]["has_raw"] = True if row[25] == 7 else False
@@ -903,17 +929,17 @@ class PhotosDB:
         # get additional details from RKMaster, needed for RAW processing
         c.execute(
             """ SELECT 
-	            RKMaster.uuid,
+                RKMaster.uuid,
                 RKMaster.volumeId, 
                 RKMaster.imagePath, 
-	            RKMaster.isMissing, 
+                RKMaster.isMissing, 
                 RKMaster.originalFileName, 
                 RKMaster.UTI,
-	            RKMaster.modelID, 
+                RKMaster.modelID, 
                 RKMaster.fileSize, 
                 RKMaster.isTrulyRaw,
-	            RKMaster.alternateMasterUuid
-	            FROM RKMaster
+                RKMaster.alternateMasterUuid
+                FROM RKMaster
             """
         )
 
@@ -1214,17 +1240,24 @@ class PhotosDB:
 
     def _build_album_folder_hierarchy_4(self, uuid, folders=None):
         """ recursively build folder/album hierarchy
-            uuid: uuid of the album/folder being processed
-            folders: dict holding the folder hierarchy """
+            uuid: parent uuid of the album being processed 
+                 (parent uuid is a folder in RKFolders)
+            folders: dict holding the folder hierarchy 
+            NOTE: This implementation is different than _build_album_folder_hierarchy_5 
+            which takes the uuid of the album being processed.  Here uuid is the parent uuid
+            of the parent folder album because in Photos <=4, folders are in RKFolders and 
+            albums in RKAlbums.  In Photos 5, folders are just special albums 
+            with kind = _PHOTOS_5_FOLDER_KIND """
 
         parent_uuid = self._dbfolder_details[uuid]["parentFolderUuid"]
 
-        # logging.warning(f"uuid = {uuid}, parent = {parent_uuid}, folders = {folders}")
-
         if parent_uuid is None:
             return folders
 
         if parent_uuid == _PHOTOS_4_TOP_LEVEL_ALBUM:
+            if not folders:
+                # this is a top-level folder with no sub-folders
+                folders = {uuid: None}
             # at top of hierarchy, we're done
             return folders
 
@@ -1485,12 +1518,18 @@ class PhotosDB:
             # There are sometimes negative values for lastmodifieddate in the database
             # I don't know what these mean but they will raise exception in datetime if
             # not accounted for
-            if row[4] is not None and row[4] >= 0:
+            try:
                 info["lastmodifieddate"] = datetime.fromtimestamp(row[4] + td)
-            else:
+            except ValueError:
+                info["lastmodifieddate"] = None
+            except TypeError:
                 info["lastmodifieddate"] = None
 
-            info["imageDate"] = datetime.fromtimestamp(row[5] + td)
+            try:
+                info["imageDate"] = datetime.fromtimestamp(row[5] + td)
+            except ValueError:
+                info["imageDate"] = datetime.date(1970, 1, 1)
+
             info["imageTimeZoneOffsetSeconds"] = row[6]
             info["hidden"] = row[9]
             info["favorite"] = row[10]
@@ -1607,7 +1646,12 @@ class PhotosDB:
             info["momentID"] = row[26]
 
             # original resource choice (e.g. RAW or jpeg)
+            # for images part of a RAW/jpeg pair,
+            # ZADDITIONALASSETATTRIBUTES.ZORIGINALRESOURCECHOICE
+            # = 0 if jpeg is selected as "original" in Photos (the default)
+            # = 1 if RAW is selected as "original" in Photos
             info["original_resource_choice"] = row[27]
+            info["raw_is_original"] = True if row[27] == 1 else False
 
             # associated RAW image info
             # will be filled in later
@@ -1820,6 +1864,12 @@ class PhotosDB:
         # close connection and remove temporary files
         conn.close()
 
+        # process search info
+        self._process_searchinfo()
+
+        # process exif info
+        self._process_exifinfo()
+
         # done processing, dump debug data if requested
         if _debug():
             logging.debug("Faces (_dbfaces_uuid):")
@@ -1878,7 +1928,7 @@ class PhotosDB:
             # folder with no parent (e.g. shared iCloud folders)
             return folders
 
-        if self._db_version >= _PHOTOS_5_VERSION and parent == self._folder_root_pk:
+        if self._db_version > _PHOTOS_4_VERSION and parent == self._folder_root_pk:
             # at the top of the folder hierarchy, we're done
             return folders
 
@@ -2156,3 +2206,7 @@ class PhotosDB:
             return self.__dict__ == other.__dict__
 
         return False
+
+    def __len__(self):
+        """ returns number of photos in the database """
+        return len(self._dbphotos)

osxphotos/placeinfo.py

@@ -493,6 +493,14 @@ class PlaceInfo4(PlaceInfo):
         strval = "PlaceInfo(" + ", ".join([f"{k}='{v}'" for k, v in info.items()]) + ")"
         return strval
 
+    def as_dict(self):
+        info = {
+            "name": self.name,
+            "names": self.names._asdict(),
+            "country_code": self.country_code,
+        }
+        return info
+
 
 class PlaceInfo5(PlaceInfo):
     """ Reverse geolocation place info for a photo (Photos >= 5) """
@@ -624,3 +632,14 @@ class PlaceInfo5(PlaceInfo):
         }
         strval = "PlaceInfo(" + ", ".join([f"{k}='{v}'" for k, v in info.items()]) + ")"
         return strval
+
+    def as_dict(self):
+        info = {
+            "name": self.name,
+            "names": self.names._asdict(),
+            "country_code": self.country_code,
+            "ishome": self.ishome,
+            "address_str": self.address_str,
+            "address": self.address._asdict(),
+        }
+        return info

osxphotos/template.py

@@ -1,439 +0,0 @@
-""" Custom template system for osxphotos """
-
-# Rolled my own template system because:
-# 1. Needed to handle multiple values (e.g. album, keyword)
-# 2. Needed to handle default values if template not found
-# 3. Didn't want user to need to know python (e.g. by using Mako which is
-#    already used elsewhere in this project)
-# 4. Couldn't figure out how to do #1 and #2 with str.format()
-#
-# This code isn't elegant but it seems to work well.  PRs gladly accepted.
-
-import datetime
-import os
-import pathlib
-import re
-from typing import Tuple, List  # pylint: disable=syntax-error
-
-from .photoinfo import PhotoInfo
-from ._constants import _UNKNOWN_PERSON
-
-# Permitted substitutions (each of these returns a single value or None)
-TEMPLATE_SUBSTITUTIONS = {
-    "{name}": "Filename of the photo",
-    "{original_name}": "Photo's original filename when imported to Photos",
-    "{title}": "Title of the photo",
-    "{descr}": "Description of the photo",
-    "{created.date}": "Photo's creation date in ISO format, e.g. '2020-03-22'",
-    "{created.year}": "4-digit year of file creation time",
-    "{created.yy}": "2-digit year of file creation time",
-    "{created.mm}": "2-digit month of the file creation time (zero padded)",
-    "{created.month}": "Month name in user's locale of the file creation time",
-    "{created.mon}": "Month abbreviation in the user's locale of the file creation time",
-    "{created.doy}": "3-digit day of year (e.g Julian day) of file creation time, starting from 1 (zero padded)",
-    "{modified.date}": "Photo's modification date in ISO format, e.g. '2020-03-22'",
-    "{modified.year}": "4-digit year of file modification time",
-    "{modified.yy}": "2-digit year of file modification time",
-    "{modified.mm}": "2-digit month of the file modification time (zero padded)",
-    "{modified.month}": "Month name in user's locale of the file modification time",
-    "{modified.mon}": "Month abbreviation in the user's locale of the file modification time",
-    "{modified.doy}": "3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)",
-    "{place.name}": "Place name from the photo's reverse geolocation data, as displayed in Photos",
-    "{place.country_code}": "The ISO country code from the photo's reverse geolocation data",
-    "{place.name.country}": "Country name from the photo's reverse geolocation data",
-    "{place.name.state_province}": "State or province name from the photo's reverse geolocation data",
-    "{place.name.city}": "City or locality name from the photo's reverse geolocation data",
-    "{place.name.area_of_interest}": "Area of interest name (e.g. landmark or public place) from the photo's reverse geolocation data",
-    "{place.address}": "Postal address from the photo's reverse geolocation data, e.g. '2007 18th St NW, Washington, DC 20009, United States'",
-    "{place.address.street}": "Street part of the postal address, e.g. '2007 18th St NW'",
-    "{place.address.city}": "City part of the postal address, e.g. 'Washington'",
-    "{place.address.state_province}": "State/province part of the postal address, e.g. 'DC'",
-    "{place.address.postal_code}": "Postal code part of the postal address, e.g. '20009'",
-    "{place.address.country}": "Country name of the postal address, e.g. 'United States'",
-    "{place.address.country_code}": "ISO country code of the postal address, e.g. 'US'",
-}
-
-# Permitted multi-value substitutions (each of these returns None or 1 or more values)
-TEMPLATE_SUBSTITUTIONS_MULTI_VALUED = {
-    "{album}": "Album(s) photo is contained in",
-    "{folder_album}": "Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder",
-    "{keyword}": "Keyword(s) assigned to photo",
-    "{person}": "Person(s) / face(s) in a photo",
-}
-
-# Just the multi-valued substitution names without the braces
-MULTI_VALUE_SUBSTITUTIONS = [
-    field.replace("{", "").replace("}", "")
-    for field in TEMPLATE_SUBSTITUTIONS_MULTI_VALUED.keys()
-]
-
-
-def get_template_value(lookup, photo):
-    """ lookup template value (single-value template substitutions) for use in make_subst_function
-        lookup: value to find a match for
-        photo: PhotoInfo object whose data will be used for value substitutions
-        returns: either the matching template value (which may be None)
-        raises: KeyError if no rule exists for lookup """
-
-    # must be a valid keyword
-    if lookup == "name":
-        return pathlib.Path(photo.filename).stem
-
-    if lookup == "original_name":
-        return pathlib.Path(photo.original_filename).stem
-
-    if lookup == "title":
-        return photo.title
-
-    if lookup == "descr":
-        return photo.description
-
-    if lookup == "created.date":
-        return DateTimeFormatter(photo.date).date
-
-    if lookup == "created.year":
-        return DateTimeFormatter(photo.date).year
-
-    if lookup == "created.yy":
-        return DateTimeFormatter(photo.date).yy
-
-    if lookup == "created.mm":
-        return DateTimeFormatter(photo.date).mm
-
-    if lookup == "created.month":
-        return DateTimeFormatter(photo.date).month
-
-    if lookup == "created.mon":
-        return DateTimeFormatter(photo.date).mon
-
-    if lookup == "created.doy":
-        return DateTimeFormatter(photo.date).doy
-
-    if lookup == "modified.date":
-        return (
-            DateTimeFormatter(photo.date_modified).date if photo.date_modified else None
-        )
-
-    if lookup == "modified.year":
-        return (
-            DateTimeFormatter(photo.date_modified).year if photo.date_modified else None
-        )
-
-    if lookup == "modified.yy":
-        return (
-            DateTimeFormatter(photo.date_modified).yy if photo.date_modified else None
-        )
-
-    if lookup == "modified.mm":
-        return (
-            DateTimeFormatter(photo.date_modified).mm if photo.date_modified else None
-        )
-
-    if lookup == "modified.month":
-        return (
-            DateTimeFormatter(photo.date_modified).month
-            if photo.date_modified
-            else None
-        )
-
-    if lookup == "modified.mon":
-        return (
-            DateTimeFormatter(photo.date_modified).mon if photo.date_modified else None
-        )
-
-    if lookup == "modified.doy":
-        return (
-            DateTimeFormatter(photo.date_modified).doy if photo.date_modified else None
-        )
-
-    if lookup == "place.name":
-        return photo.place.name if photo.place else None
-
-    if lookup == "place.country_code":
-        return photo.place.country_code if photo.place else None
-
-    if lookup == "place.name.country":
-        return (
-            photo.place.names.country[0]
-            if photo.place and photo.place.names.country
-            else None
-        )
-
-    if lookup == "place.name.state_province":
-        return (
-            photo.place.names.state_province[0]
-            if photo.place and photo.place.names.state_province
-            else None
-        )
-
-    if lookup == "place.name.city":
-        return (
-            photo.place.names.city[0]
-            if photo.place and photo.place.names.city
-            else None
-        )
-
-    if lookup == "place.name.area_of_interest":
-        return (
-            photo.place.names.area_of_interest[0]
-            if photo.place and photo.place.names.area_of_interest
-            else None
-        )
-
-    if lookup == "place.address":
-        return (
-            photo.place.address_str if photo.place and photo.place.address_str else None
-        )
-
-    if lookup == "place.address.street":
-        return (
-            photo.place.address.street
-            if photo.place and photo.place.address.street
-            else None
-        )
-
-    if lookup == "place.address.city":
-        return (
-            photo.place.address.city
-            if photo.place and photo.place.address.city
-            else None
-        )
-
-    if lookup == "place.address.state_province":
-        return (
-            photo.place.address.state_province
-            if photo.place and photo.place.address.state_province
-            else None
-        )
-
-    if lookup == "place.address.postal_code":
-        return (
-            photo.place.address.postal_code
-            if photo.place and photo.place.address.postal_code
-            else None
-        )
-
-    if lookup == "place.address.country":
-        return (
-            photo.place.address.country
-            if photo.place and photo.place.address.country
-            else None
-        )
-
-    if lookup == "place.address.country_code":
-        return (
-            photo.place.address.iso_country_code
-            if photo.place and photo.place.address.iso_country_code
-            else None
-        )
-
-    # if here, didn't get a match
-    raise KeyError(f"No rule for processing {lookup}")
-
-
-def render_filepath_template(template, photo, none_str="_"):
-    """ render a filename or directory template 
-        template: str template 
-        photo: PhotoInfo object
-        none_str: str to use default for None values, default is '_' """
-
-    # the rendering happens in two phases:
-    # phase 1: handle all the single-value template substitutions
-    #          results in a single string with all the template fields replaced
-    # phase 2: loop through all the multi-value template substitutions
-    #          could result in multiple strings
-    #          e.g. if template is "{album}/{person}" and there are 2 albums and 3 persons in the photo
-    #          there would be 6 possible renderings (2 albums x 3 persons)
-
-    # regex to find {template_field,optional_default} in strings
-    # for explanation of regex see https://regex101.com/r/4JJg42/1
-    # pylint: disable=anomalous-backslash-in-string
-    regex = r"(?<!\{)\{([^\\,}]+)(,{0,1}(([\w\-. ]+))?)(?=\}(?!\}))\}"
-
-    if type(template) is not str:
-        raise TypeError(f"template must be type str, not {type(template)}")
-
-    if type(photo) is not PhotoInfo:
-        raise TypeError(f"photo must be type osxphotos.PhotoInfo, not {type(photo)}")
-
-    def make_subst_function(photo, none_str, get_func=get_template_value):
-        """ returns: substitution function for use in re.sub 
-            photo: a PhotoInfo object
-            none_str: value to use if substitution lookup is None and no default provided
-            get_func: function that gets the substitution value for a given template field
-                      default is get_template_value which handles the single-value fields """
-
-        # closure to capture photo, none_str in subst
-        def subst(matchobj):
-            groups = len(matchobj.groups())
-            if groups == 4:
-                try:
-                    val = get_func(matchobj.group(1), photo)
-                except KeyError:
-                    return matchobj.group(0)
-
-                if val is None:
-                    return (
-                        matchobj.group(3) if matchobj.group(3) is not None else none_str
-                    )
-                else:
-                    return val
-            else:
-                raise ValueError(
-                    f"Unexpected number of groups: expected 4, got {groups}"
-                )
-
-        return subst
-
-    subst_func = make_subst_function(photo, none_str)
-
-    # do the replacements
-    rendered = re.sub(regex, subst_func, template)
-
-    # do multi-valued placements
-    # start with the single string from phase 1 above then loop through all
-    # multi-valued fields and all values for each of those fields
-    # rendered_strings will be updated as each field is processed
-    # for example: if two albums, two keywords, and one person and template is:
-    # "{created.year}/{album}/{keyword}/{person}"
-    # rendered strings would do the following:
-    # start (created.year filled in phase 1)
-    #   ['2011/{album}/{keyword}/{person}']
-    # after processing albums:
-    #   ['2011/Album1/{keyword}/{person}',
-    #    '2011/Album2/{keyword}/{person}',]
-    # after processing keywords:
-    #   ['2011/Album1/keyword1/{person}',
-    #    '2011/Album1/keyword2/{person}',
-    #    '2011/Album2/keyword1/{person}',
-    #    '2011/Album2/keyword2/{person}',]
-    # after processing person:
-    #   ['2011/Album1/keyword1/person1',
-    #    '2011/Album1/keyword2/person1',
-    #    '2011/Album2/keyword1/person1',
-    #    '2011/Album2/keyword2/person1',]
-
-    rendered_strings = set([rendered])
-    for field in MULTI_VALUE_SUBSTITUTIONS:
-        if field == "album":
-            values = photo.albums
-        elif field == "keyword":
-            values = photo.keywords
-        elif field == "person":
-            values = photo.persons
-            # remove any _UNKNOWN_PERSON values
-            values = [val for val in values if val != _UNKNOWN_PERSON]
-        elif field == "folder_album":
-            values = []
-            # photos must be in an album to be in a folder
-            for album in photo.album_info:
-                if album.folder_names:
-                    # album in folder
-                    folder = os.path.sep.join(album.folder_names)
-                    folder += os.path.sep + album.title
-                    values.append(folder)
-                else:
-                    # album not in folder
-                    values.append(album.title)
-
-        else:
-            raise ValueError(f"Unhandleded template value: {field}")
-
-        # If no values, insert None so code below will substite none_str for None
-        values = values or [None]
-
-        # Build a regex that matches only the field being processed
-        re_str = r"(?<!\\)\{(" + field + r")(,{0,1}(([\w\-. ]+))?)\}"
-        regex_multi = re.compile(re_str)
-
-        # holds each of the new rendered_strings, set() to avoid duplicates
-        new_strings = set()
-
-        for str_template in rendered_strings:
-            for val in values:
-
-                def get_template_value_multi(lookup_value, photo):
-                    """ Closure passed to make_subst_function get_func 
-                        Capture val and field in the closure 
-                        Allows make_subst_function to be re-used w/o modification """
-                    if lookup_value == field:
-                        return val
-                    else:
-                        raise KeyError(f"Unexpected value: {lookup_value}")
-
-                subst = make_subst_function(
-                    photo, none_str, get_func=get_template_value_multi
-                )
-                new_string = regex_multi.sub(subst, str_template)
-                new_strings.add(new_string)
-
-        # update rendered_strings for the next field to process
-        rendered_strings = new_strings
-
-    # find any {fields} that weren't replaced
-    unmatched = []
-    for rendered_str in rendered_strings:
-        unmatched.extend(
-            [
-                no_match[0]
-                for no_match in re.findall(regex, rendered_str)
-                if no_match[0] not in unmatched
-            ]
-        )
-
-    # fix any escaped curly braces
-    rendered_strings = [
-        rendered_str.replace("{{", "{").replace("}}", "}")
-        for rendered_str in rendered_strings
-    ]
-
-    return rendered_strings, unmatched
-
-
-class DateTimeFormatter:
-    """ provides property access to formatted datetime.datetime strftime values """
-
-    def __init__(self, dt: datetime.datetime):
-        self.dt = dt
-
-    @property
-    def date(self):
-        """ ISO date in form 2020-03-22 """
-        date = self.dt.date().isoformat()
-        return date
-
-    @property
-    def year(self):
-        """ 4 digit year """
-        year = f"{self.dt.year}"
-        return year
-
-    @property
-    def yy(self):
-        """ 2 digit year """
-        yy = f"{self.dt.strftime('%y')}"
-        return yy
-
-    @property
-    def mm(self):
-        """ 2 digit month """
-        mm = f"{self.dt.strftime('%m')}"
-        return mm
-
-    @property
-    def month(self):
-        """ Month as locale's full name """
-        month = f"{self.dt.strftime('%B')}"
-        return month
-
-    @property
-    def mon(self):
-        """ Month as locale's abbreviated name """
-        mon = f"{self.dt.strftime('%b')}"
-        return mon
-
-    @property
-    def doy(self):
-        """ Julian day of year starting from 001 """
-        doy = f"{self.dt.strftime('%j')}"
-        return doy

osxphotos/templates/xmp_sidecar.mako

@@ -79,16 +79,16 @@
             xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
             ${dc_description(photo.description)}
             ${dc_title(photo.title)}
-            ${dc_subject(photo.keywords + photo.persons)}
+            ${dc_subject(subjects)}
             ${dc_datecreated(photo.date)}
         </rdf:Description>
         <rdf:Description rdf:about='' 
             xmlns:Iptc4xmpExt='http://iptc.org/std/Iptc4xmpExt/2008-02-29/'>
-            ${iptc_personinimage(photo.persons)}
+            ${iptc_personinimage(persons)}
         </rdf:Description>
         <rdf:Description rdf:about='' 
             xmlns:digiKam='http://www.digikam.org/ns/1.0/'>
-            ${dk_tagslist(photo.keywords)}
+            ${dk_tagslist(keywords)}
         </rdf:Description>
         <rdf:Description rdf:about='' 
             xmlns:xmp='http://ns.adobe.com/xap/1.0/'>

osxphotos/utils.py

@@ -118,6 +118,26 @@ def _dd_to_dms(dd):
     return int(deg_), int(min_), sec_
 
 
+def _hardlink_file(src, dest):
+    """ Hardlinks a file from src path to dest path 
+        src: source path as string 
+        dest: destination path as string
+        Raises exception if linking fails or either path is None """
+
+    if src is None or dest is None:
+        raise ValueError("src and dest must not be None", src, dest)
+
+    if not os.path.isfile(src):
+        raise FileNotFoundError("src file does not appear to exist", src)
+
+    # if error on copy, subprocess will raise CalledProcessError
+    try:
+        os.link(src, dest)
+    except Exception as e:
+        logging.critical(f"os.link returned error: {e}")
+        raise e
+
+
 def _copy_file(src, dest, norsrc=False):
     """ Copies a file from src path to dest path 
         src: source path as string 
@@ -492,3 +512,30 @@ def _db_is_locked(dbname):
         locked = True
 
     return locked
+
+
+# OSXPHOTOS_XATTR_UUID = "com.osxphotos.uuid"
+
+# def get_uuid_for_file(filepath):
+#     """ returns UUID associated with an exported file
+#         filepath: path to exported photo
+#     """
+#     attr = xattr.xattr(filepath)
+#     try:
+#         uuid_bytes = attr[OSXPHOTOS_XATTR_UUID]
+#         uuid_str = uuid_bytes.decode('utf-8')
+#     except KeyError:
+#         uuid_str = None
+#     return uuid_str
+    
+# def set_uuid_for_file(filepath, uuid):
+#     """ sets the UUID associated with an exported file
+#         filepath: path to exported photo
+#         uuid: uuid string for photo
+#     """
+#     if not os.path.exists(filepath):
+#         raise FileNotFoundError(f"Missing file: {filepath}")
+
+#     attr = xattr.xattr(filepath)
+#     uuid_bytes = bytes(uuid, 'utf-8')
+#     attr.set(OSXPHOTOS_XATTR_UUID, uuid_bytes)

setup.py

@@ -3,7 +3,7 @@
 #
 # setup.py script for osxphotos
 #
-# Copyright (c) 2019 Rhet Turnbull, rturnbull+git@gmail.com
+# Copyright (c) 2019, 2020 Rhet Turnbull, rturnbull+git@gmail.com
 # All rights reserved.
 #
 # Permission is hereby granted, free of charge, to any person
@@ -27,23 +27,43 @@
 # SOFTWARE.
 
 import os
+import platform
+
 from setuptools import find_packages, setup
 
-this_directory = os.path.abspath(os.path.dirname(__file__))
-with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
-    long_description = f.read()
+# python version as 2-digit float (e.g. 3.6)
+py_ver = float(".".join(platform.python_version_tuple()[:2]))
 
+# holds config info read from disk
 about = {}
+this_directory = os.path.abspath(os.path.dirname(__file__))
+
+# get version info from _version
 with open(
     os.path.join(this_directory, "osxphotos", "_version.py"), mode="r", encoding="utf-8"
 ) as f:
     exec(f.read(), about)
 
+# read README.md into long_description
+with open(os.path.join(this_directory, "README.md"), encoding="utf-8") as f:
+    about["long_description"] = f.read()
+
+# ugly hack to install custom version of bpylist2 needed for Python < 3.8
+# the stock version of bylist2==2.0.3 causes an error related to
+# "pkg_resources.ContextualVersionConflict: (pycodestyle 2.3.1..."
+# PEP 508 no help here as URL-based lookups not allowed in PyPI packages
+# if you know a better way, PRs welcome!
+# once I go to 3.8+ required, this won't be necessary as bpylist2 3.0+ solves this issue
+if py_ver < 3.8:
+    os.system(
+        "python3 -m pip install git+git://github.com/RhetTbull/bpylist2.git#egg=bpylist2"
+    )
+
 setup(
     name="osxphotos",
     version=about["__version__"],
     description="Manipulate (read-only) Apple's Photos app library on Mac OS X",
-    long_description=long_description,
+    long_description=about["long_description"],
     long_description_content_type="text/markdown",
     author="Rhet Turnbull",
     author_email="rturnbull+git@gmail.com",
@@ -58,7 +78,7 @@ setup(
         "Intended Audience :: Developers",
         "License :: OSI Approved :: MIT License",
         "Operating System :: MacOS :: MacOS X",
-        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.8",
         "Topic :: Software Development :: Libraries :: Python Modules",
     ],
     install_requires=[
@@ -69,6 +89,7 @@ setup(
         "bpylist2==2.0.3;python_version<'3.8'",
         "bpylist2==3.0.0;python_version>='3.8'",
         "pathvalidate==2.2.1",
+        "dataclasses==0.7;python_version<'3.7'",
     ],
     entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]},
     include_package_data=True,

tests/README.md

@@ -1,14 +1,22 @@
 # Tests for osxphotos #
 
 ## Running Tests ##
-Tests require pytest:
+Tests require pytest and pytest-mock:
 `pip install pytest`
+`pip install pytest-mock`
 
 To run the tests, do the following from the main source folder:
 `python -m pytest tests/`
 
 Running the tests this way allows the library to be tested without installing it.
 
+## Skipped Tests ##
+A few tests will look for certain environment variables to determine if they should run.
+
+Some of the export tests rely on photos in my local library and will look for `OSXPHOTOS_TEST_EXPORT=1` to determine if they should run.
+
+One test for locale does not run on GitHub's automated workflow and will look for `OSXPHOTOS_TEST_LOCALE=1` to determine if it should be run.  If you want to run this test, set the environment variable.  
+
 ## Attribution ##
 These tests utilize a test Photos library. The test library is populated with photos from [flickr](https://www.flickr.com).  All images used are licensed under Creative Commons 2.0 Attribution [license](https://creativecommons.org/licenses/by/2.0/).  
 

tests/Test-10.13.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -8,8 +8,11 @@
 		<array/>
 		<key>ExpandedSidebarItemIdentifiers</key>
 		<array>
+			<string>obfeGcvoT1auxoh2Tu86OQ</string>
 			<string>TopLevelAlbums</string>
 			<string>TopLevelSlideshows</string>
+			<string>MBS8+gBrQCWQxmcav+C8HQ</string>
+			<string>cHwwVoUiQ8a2nZNXgVsnCA</string>
 		</array>
 		<key>IPXWorkspaceControllerZoomLevelsKey</key>
 		<dict>
@@ -23,11 +26,11 @@
 			<key>key</key>
 			<integer>1</integer>
 			<key>lastKnownDisplayName</key>
-			<string>September 28, 2018</string>
+			<string>Pumpkin Farm (1)</string>
 			<key>type</key>
 			<string>album</string>
 			<key>uuid</key>
-			<string>+Ep8CrNRRhea9eVA618FMg</string>
+			<string>AU8Gp8bwRlOvngZFgwXBdg</string>
 		</dict>
 		<key>lastKnownItemCounts</key>
 		<dict>

tests/Test-10.13.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2019-07-28T01:23:52Z</date>
+	<date>2020-04-30T14:09:38Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2019-07-28T01:23:52Z</date>
+	<date>2020-05-01T04:27:48Z</date>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/private/com.apple.photoanalysisd/VisionService/PersonPromoter

@@ -2,7 +2,11 @@
 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0">
 <dict>
+	<key>ProcessedInQuiescentState</key>
+	<true/>
 	<key>SuggestedMeIdentifier</key>
 	<string></string>
+	<key>Version</key>
+	<integer>3</integer>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/private/com.apple.photomodel/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2019-07-27T12:01:15Z</date>
+		<date>2020-05-01T03:34:50Z</date>
 	</dict>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2019-07-26T20:15:18Z</date>
+	<date>2020-04-30T12:51:41Z</date>
 </dict>
 </plist>

tests/Test-10.13.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>615</integer>
+	<integer>670</integer>
 	<key>LibraryBuildTag</key>
 	<string>BEA5F0E8-BA6B-4462-8F73-3E53BBE4C943</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.13.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>615</integer>
+		<integer>670</integer>
 		<key>LibraryBuildTag</key>
 		<string>BEA5F0E8-BA6B-4462-8F73-3E53BBE4C943</string>
 		<key>LibrarySchemaVersion</key>
@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-26T20:15:17Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2019-07-27T12:01:15Z</date>
+	<date>2020-05-01T03:34:50Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>

tests/Test-10.14.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-04-18T18:01:02Z</date>
+	<date>2020-04-25T23:54:43Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-04-18T17:22:55Z</date>
+	<date>2020-04-26T06:26:10Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2020-04-17T17:49:52Z</date>
+	<date>2020-04-25T23:54:29Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/recovery/Info.plist

@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-27T13:16:43Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2020-04-17T17:51:16Z</date>
+	<date>2020-04-25T23:56:35Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>

tests/Test-10.15.1.photoslibrary/database/Photos.sqlite.lock

@@ -7,7 +7,7 @@
 	<key>hostuuid</key>
 	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
 	<key>pid</key>
-	<integer>900</integer>
+	<integer>725</integer>
 	<key>processname</key>
 	<string>photolibraryd</string>
 	<key>uid</key>

tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist

@@ -3,24 +3,24 @@
 <plist version="1.0">
 <dict>
 	<key>BackgroundHighlightCollection</key>
-	<date>2020-04-17T14:33:32Z</date>
+	<date>2020-05-01T23:03:12Z</date>
 	<key>BackgroundHighlightEnrichment</key>
-	<date>2020-04-17T14:33:32Z</date>
+	<date>2020-05-01T23:03:11Z</date>
 	<key>BackgroundJobAssetRevGeocode</key>
-	<date>2020-04-17T14:33:33Z</date>
+	<date>2020-05-02T01:35:19Z</date>
 	<key>BackgroundJobSearch</key>
-	<date>2020-04-17T14:33:33Z</date>
+	<date>2020-05-01T23:03:12Z</date>
 	<key>BackgroundPeopleSuggestion</key>
-	<date>2020-04-17T14:33:31Z</date>
+	<date>2020-05-01T23:03:11Z</date>
 	<key>BackgroundUserBehaviorProcessor</key>
-	<date>2020-04-17T07:32:04Z</date>
+	<date>2020-05-01T23:03:13Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundGraphConsistencyUpdateJobDateKey</key>
-	<date>2020-04-17T14:33:37Z</date>
+	<date>2020-05-02T01:35:36Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-04-17T07:32:00Z</date>
+	<date>2020-05-01T23:03:11Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-04-17T14:33:34Z</date>
+	<date>2020-05-02T01:35:19Z</date>
 	<key>SiriPortraitDonation</key>
-	<date>2020-04-17T07:32:04Z</date>
+	<date>2020-05-01T23:03:13Z</date>
 </dict>
 </plist>

tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>FaceIDModelLastGenerationKey</key>
-	<date>2020-04-17T07:32:07Z</date>
+	<date>2020-05-01T23:03:14Z</date>
 	<key>LastContactClassificationKey</key>
-	<date>2020-04-17T07:32:12Z</date>
+	<date>2020-05-01T23:03:18Z</date>
 </dict>
 </plist>

tests/Test-10.15.4.photoslibrary/database/Photos.sqlite.lock

@@ -7,7 +7,7 @@
 	<key>hostuuid</key>
 	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
 	<key>pid</key>
-	<integer>3212</integer>
+	<integer>3731</integer>
 	<key>processname</key>
 	<string>photolibraryd</string>
 	<key>uid</key>