test library update

Update README.md

CHANGELOG.md

@@ -4,6 +4,54 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.28.1](https://github.com/RhetTbull/osxphotos/compare/v0.27.4...v0.28.1)
+
+> 18 April 2020
+
+- Initial work on suppport for associated RAW images [`7e42ebb`](https://github.com/RhetTbull/osxphotos/commit/7e42ebb2402d45cd5d20bdd55bddddaa9db4679f)
+- Initial support for RAW photos in Photos 4 to address issue #101 [`9d15147`](https://github.com/RhetTbull/osxphotos/commit/9d151478d610291b8d482aafae3d445dfd391fca)
+- replaced CLI option --original-name with --current-name [`36c2821`](https://github.com/RhetTbull/osxphotos/commit/36c2821a0fa62eaaa54cf1edc2d9c6da98155354)
+
+#### [v0.27.4](https://github.com/RhetTbull/osxphotos/compare/v0.27.3...v0.27.4)
+
+> 12 April 2020
+
+- Added {folder_album} to template and --folder to CLI [`b7c7b9f`](https://github.com/RhetTbull/osxphotos/commit/b7c7b9f0664e69c743bdd8a228ad2936cf6b7600)
+- Test library update [`21e7020`](https://github.com/RhetTbull/osxphotos/commit/21e7020fec406b0f3926d7adc8a1451bfe77e75a)
+- Updated CHANGELOG.md [`952741d`](https://github.com/RhetTbull/osxphotos/commit/952741d488d2fbbaf8a0c1d3781ad7c4205c068f)
+
+#### [v0.27.3](https://github.com/RhetTbull/osxphotos/compare/v0.27.1...v0.27.3)
+
+> 12 April 2020
+
+- Added additional tests for album_info [`97362fc`](https://github.com/RhetTbull/osxphotos/commit/97362fc0f13b2867abc013f4ba97ae60b0700894)
+- Fixed bug with handling of deleted albums [`9fef12e`](https://github.com/RhetTbull/osxphotos/commit/9fef12ed37634a7bdb11232976b4b2ddccd1a7cb)
+
+#### [v0.27.1](https://github.com/RhetTbull/osxphotos/compare/v0.27.0...v0.27.1)
+
+> 12 April 2020
+
+- Changed AlbumInfo and FolderInfo interface to maintain backwards compatibility with PhotosDB.albums [`e09f0b4`](https://github.com/RhetTbull/osxphotos/commit/e09f0b40f1671d70ee399cdc519492b04fac8adc)
+- Updated CHANGELOG.md [`b749681`](https://github.com/RhetTbull/osxphotos/commit/b749681c6d2545eacf653ab1b2a5d1384e3123eb)
+
+#### [v0.27.0](https://github.com/RhetTbull/osxphotos/compare/v0.26.1...v0.27.0)
+
+> 11 April 2020
+
+- Update README.md [`#95`](https://github.com/RhetTbull/osxphotos/pull/95)
+- Added tests and README for AlbumInfo and FolderInfo [`d6a22b7`](https://github.com/RhetTbull/osxphotos/commit/d6a22b765ab17f6ef1ba8c50b77946f090979968)
+- Added albuminfo.py for AlbumInfo and FolderInfo classes [`9636572`](https://github.com/RhetTbull/osxphotos/commit/96365728c2ff42abfb6828872ffac53b4c3c8024)
+- Updated CHANGELOG.md [`cde56e9`](https://github.com/RhetTbull/osxphotos/commit/cde56e9d13baf3098ec85839cf1aaa33b4915ac9)
+- Update README.md TOC [`8544667`](https://github.com/RhetTbull/osxphotos/commit/8544667c729ea0d7fe39671d909e09cda519e250)
+
+#### [v0.26.1](https://github.com/RhetTbull/osxphotos/compare/v0.26.0...v0.26.1)
+
+> 11 April 2020
+
+- Bug fix for PhotosDB.photos() query [`1c9da5e`](https://github.com/RhetTbull/osxphotos/commit/1c9da5ed6ffa21f0577906b65b7da08951725d1f)
+- Updated test library [`d74f7f4`](https://github.com/RhetTbull/osxphotos/commit/d74f7f499bf59f37ec81cfa9d49cbbf3aafb5961)
+- Updated CHANGELOG.md [`c85bb02`](https://github.com/RhetTbull/osxphotos/commit/c85bb023042e072d6688060eb259156c2fa579b9)
+
 #### [v0.26.0](https://github.com/RhetTbull/osxphotos/compare/v0.25.1...v0.26.0)
 
 > 11 April 2020

README.md

@@ -13,12 +13,15 @@
   * [Package Interface](#package-interface)
     + [PhotosDB](#photosdb)
     + [PhotoInfo](#photoinfo)
+    + [AlbumInfo](#albuminfo)
+    + [FolderInfo](#folderinfo)
     + [PlaceInfo](#placeinfo)
     + [Template Functions](#template-functions)
     + [Utility Functions](#utility-functions)
   * [Examples](#examples)
   * [Related Projects](#related-projects)
   * [Contributing](#contributing)
+  * [Known Bugs](#known-bugs)
   * [Implementation Notes](#implementation-notes)
   * [Dependencies](#dependencies)
   * [Acknowledgements](#acknowledgements)
@@ -93,7 +96,10 @@ Usage: osxphotos export [OPTIONS] [PHOTOS_LIBRARY]... DEST
   Optionally, query the Photos database using 1 or more search options;  if
   more than one option is provided, they are treated as "AND"  (e.g. search
   for photos matching all options). If no query options are provided, all
-  photos will be exported.
+  photos will be exported. By default, all versions of all photos will be
+  exported including edited versions, live photo movies, burst photos, and
+  associated RAW images.  See --skip-edited, --skip-live, --skip-bursts, and
+  --skip-raw options to modify this behavior.
 
 Options:
   --db <Photos database path>     Specify Photos database path. Path to Photos
@@ -105,16 +111,22 @@ Options:
                                   order: 1. last opened library, 2. system
                                   library, 3. ~/Pictures/Photos
                                   Library.photoslibrary
-  --keyword KEYWORD               Search for keyword KEYWORD. If more than one
-                                  keyword, treated as "OR", e.g. find photos
-                                  match any keyword
-  --person PERSON                 Search for person PERSON. If more than one
-                                  person, treated as "OR", e.g. find photos
-                                  match any person
-  --album ALBUM                   Search for album ALBUM. If more than one
-                                  album, treated as "OR", e.g. find photos
-                                  match any album
-  --uuid UUID                     Search for UUID(s).
+  -V, --verbose                   Print verbose output.
+  --keyword KEYWORD               Search for photos with keyword KEYWORD. If
+                                  more than one keyword, treated as "OR", e.g.
+                                  find photos match any keyword
+  --person PERSON                 Search for photos with person PERSON. If
+                                  more than one person, treated as "OR", e.g.
+                                  find photos match any person
+  --album ALBUM                   Search for photos in album ALBUM. If more
+                                  than one album, treated as "OR", e.g. find
+                                  photos match any album
+  --folder FOLDER                 Search for photos in an album in folder
+                                  FOLDER. If more than one folder, treated as
+                                  "OR", e.g. find photos in any FOLDER.  Only
+                                  searches top level folders (e.g. does not
+                                  look at subfolders)
+  --uuid UUID                     Search for photos with UUID(s).
   --title TITLE                   Search for TITLE in title of photo.
   --no-title                      Search for photos with no title.
   --description DESC              Search for DESC in description of photo.
@@ -164,6 +176,8 @@ Options:
   --not-selfie                    Search for photos that are not selfies.
   --panorama                      Search for panorama photos.
   --not-panorama                  Search for photos that are not panoramas.
+  --has-raw                       Search for photos with both a jpeg and RAW
+                                  version
   --only-movies                   Search only for movies (default searches
                                   both images and movies).
   --only-photos                   Search only for photos/images (default
@@ -176,7 +190,6 @@ Options:
                                   Search by end item date, e.g.
                                   2000-01-12T12:00:00 or 2000-12-31 (ISO 8601
                                   w/o TZ).
-  -V, --verbose                   Print verbose output.
   --overwrite                     Overwrite existing files. Default behavior
                                   is to add (1), (2), etc to filename if file
                                   already exists. Use this with caution as it
@@ -185,19 +198,23 @@ Options:
   --export-by-date                Automatically create output folders to
                                   organize photos by date created (e.g.
                                   DEST/2019/12/20/photoname.jpg).
-  --export-edited                 Also export edited version of photo if an
-                                  edited version exists.  Edited photo will be
-                                  named in form of "photoname_edited.ext"
-  --export-bursts                 If a photo is a burst photo export all
-                                  associated burst images in the library.  Not
-                                  currently compatible with --download-
-                                  misssing; see note on --download-missing.
-  --export-live                   If a photo is a live photo export the
-                                  associated live video component.  Live video
-                                  will have same name as photo but with .mov
-                                  extension.
-  --original-name                 Use photo's original filename instead of
-                                  current filename for export.
+  --skip-edited                   Do not export edited version of photo if an
+                                  edited version exists.
+  --skip-bursts                   Do not export all associated burst images in
+                                  the library if a photo is a burst photo.
+  --skip-live                     Do not export the associated live video
+                                  component of a live photo.
+  --skip-raw                      Do not export associated RAW images of a
+                                  RAW/jpeg pair.  Note: this does not skip RAW
+                                  photos if the RAW photo does not have an
+                                  associated jpeg image (e.g. the RAW file was
+                                  imported to Photos without a jpeg preview).
+  --current-name                  Use photo's current filename instead of
+                                  original filename for export.  Note:
+                                  Starting with Photos 5, all photos are
+                                  renamed upon import.  By default, photos are
+                                  exported with the the original name they had
+                                  before import.
   --sidecar FORMAT                Create sidecar for each photo exported;
                                   valid FORMAT values: xmp, json; --sidecar
                                   json: create JSON sidecar useable by
@@ -217,9 +234,9 @@ Options:
                                   exist on disk.  This will be slow and will
                                   require internet connection. This obviously
                                   only works if the Photos library is synched
-                                  to iCloud.  Note: --download-missing is not
-                                  currently compatabile with --export-bursts;
-                                  only the primary photo will be exported--
+                                  to iCloud.  Note: --download-missing does
+                                  not currently export all burst images; only
+                                  the primary photo will be exported--
                                   associated burst images will be skipped.
   --exiftool                      Use exiftool to write metadata directly to
                                   exported photos. To use this option,
@@ -230,6 +247,11 @@ Options:
                                   output directory in the form
                                   '{name,DEFAULT}'. See below for additional
                                   details on templating system.
+  --no-extended-attributes        Don't copy extended attributes when
+                                  exporting.  You only need this if exporting
+                                  to a filesystem that doesn't support Mac OS
+                                  extended attributes.  Only use this if you
+                                  get an error while exporting.
   -h, --help                      Show this message and exit.
 
 **Templating System**
@@ -332,19 +354,22 @@ exported, one to each directory.  For example: --directory
 of the following directories if the photos were created in 2019 and were in
 albums 'Vacation' and 'Family': 2019/Vacation, 2019/Family
 
-Substitution  Description
-{album}       Album(s) photo is contained in
-{keyword}     Keyword(s) assigned to photo
-{person}      Person(s) / face(s) in a photo
+Substitution    Description
+{album}         Album(s) photo is contained in
+{folder_album}  Folder path + album photo is contained in. e.g.
+                'Folder/Subfolder/Album' or just 'Album' if no enclosing
+                folder
+{keyword}       Keyword(s) assigned to photo
+{person}        Person(s) / face(s) in a photo
 ```
 
-Example: export all photos to ~/Desktop/export, including edited versions and live photo movies, group in folders by date created
+Example: export all photos to ~/Desktop/export group in folders by date created
 
-`osxphotos export --export-edited --export-live --export-by-date ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
+`osxphotos export --export-by-date ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
 
 **Note**: Photos library/database path can also be specified using --db option:
 
-`osxphotos export --export-edited --export-live --export-by-date --db ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
+`osxphotos export --export-by-date --db ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
 
 Example: find all photos with keyword "Kids" and output results to json file named results.json:
 
@@ -554,22 +579,52 @@ keywords = photosdb.keywords
 
 Returns a list of the keywords found in the Photos library
 
-#### `album_names`
+#### `album_info`
 ```python
 # assumes photosdb is a PhotosDB object (see above)
-albums = photosdb.album_names
+albums = photosdb.album_info
 ```
 
-Returns a list of the albums found in the Photos library.  
+Returns a list of [AlbumInfo](#AlbumInfo) objects representing albums in the database or empty list if there are no albums.  See also [albums](#albums).
+
+#### `albums`
+```python
+# assumes photosdb is a PhotosDB object (see above)
+album_names = photosdb.albums
+```
+
+Returns a list of the album names found in the Photos library.  
 
 **Note**: In Photos 5.0 (MacOS 10.15/Catalina), It is possible to have more than one album with the same name in Photos.  Albums with duplicate names are treated as a single album and the photos in each are combined.  For example, if you have two albums named "Wedding" and each has 2 photos, osxphotos will treat this as a single album named "Wedding" with 4 photos in it.
 
-#### `album_names_shared`
+See also [album_info](#album_info.)
 
-Returns list of shared albums found in photos database (e.g. albums shared via iCloud photo sharing)
+#### `albums_shared`
+
+Returns list of shared album names found in photos database (e.g. albums shared via iCloud photo sharing)
 
 **Note**: *Only valid for Photos 5 / MacOS 10.15*; on Photos <= 4, prints warning and returns empty list.
 
+#### `folder_info`
+```python
+# assumes photosdb is a PhotosDB object (see above)
+folders = photosdb.folder_info
+```
+
+Returns a list of [FolderInfo](#FolderInfo) objects representing top level folders in the database or empty list if there are no folders.  See also [folders](#folders).
+
+**Note**: Currently folder_info is only implemented for Photos 5 (Catalina); will return empty list and output warning if called on earlier database versions.
+
+#### `folders`
+```python
+# assumes photosdb is a PhotosDB object (see above)
+folders = photosdb.folders
+```
+
+Returns a list names of top level folder names in the database. 
+
+**Note**: Currently folders is only implemented for Photos 5 (Catalina); will return empty list and output warning if called on earlier database versions.
+
 #### `persons`
 ```python
 # assumes photosdb is a PhotosDB object (see above)
@@ -779,7 +834,10 @@ Returns the title of the photo
 Returns a list of keywords (e.g. tags) applied to the photo
 
 #### `albums`
-Returns a list of albums the photo is contained in
+Returns a list of albums the photo is contained in. See also [album_info](#album_info).
+
+#### `album_info`
+Returns a list of [AlbumInfo](#AlbumInfo) objects representing the albums the photo is contained in.  See also [albums](#albums).
 
 #### `persons`
 Returns a list of the names of the persons in the photo
@@ -926,7 +984,83 @@ Then
 
 If overwrite=False and increment=False, export will fail if destination file already exists
 
-**Implementation Note**: Because the usual python file copy methods don't preserve all the metadata available on MacOS, export uses /usr/bin/ditto to do the copy for export. ditto preserves most metadata such as extended attributes, permissions, ACLs, etc.
+**Implementation Note**: Because the usual python file copy methods don't preserve all the metadata available on MacOS, export uses `/usr/bin/ditto` to do the copy for export. ditto preserves most metadata such as extended attributes, permissions, ACLs, etc.
+
+### AlbumInfo
+PhotosDB.album_info and PhotoInfo.album_info return a list of AlbumInfo objects.  Each AlbumInfo object represents a single album in the Photos library.
+
+#### `uuid`
+Returns the universally unique identifier (uuid) of the album.  This is how Photos keeps track of individual objects within the database.
+
+#### `title`
+Returns the title or name of the album.
+
+#### `photos`
+Returns a list of [PhotoInfo](#PhotoInfo) objects representing each photo contained in the album.
+
+#### `folder_list`
+Returns a hierarchical list of [FolderInfo](#FolderInfo) objects representing the folders the album is contained in.  For example, if album "AlbumInFolder" is in SubFolder2 of Folder1 as illustrated below, would return a list of `FolderInfo` objects representing ["Folder1", "SubFolder2"] 
+
+```txt
+Photos Library
+├── Folder1
+    ├── SubFolder1
+    ├── SubFolder2
+        └── AlbumInFolder
+```
+
+#### `folder_names`
+Returns a hierarchical list of names of the folders the album is contained in.  For example, if album is in SubFolder2 of Folder1 as illustrated below, would return ["Folder1", "SubFolder2"].  
+
+```txt
+Photos Library
+├── Folder1
+    ├── SubFolder1
+    ├── SubFolder2
+        └── AlbumInFolder
+```
+
+#### `parent`
+Returns a [FolderInfo](#FolderInfo) object representing the albums parent folder or `None` if album is not a in a folder.
+
+
+### FolderInfo 
+PhotosDB.folder_info returns a list of FolderInfo objects representing the top level folders in the library.  Each FolderInfo object represents a single folder in the Photos library.
+
+#### `uuid`
+Returns the universally unique identifier (uuid) of the folder.  This is how Photos keeps track of individual objects within the database.
+
+#### `title`
+Returns the title or name of the folder.
+
+#### `album_info`
+Returns a list of [AlbumInfo](#AlbumInfo) objects representing each album contained in the folder.
+
+#### `subfolders`
+Returns a list of [FolderInfo](#FolderInfo) objects representing the sub-folders of the folder.  
+
+#### `parent`
+Returns a [FolderInfo](#FolderInfo) object representing the folder's parent folder or `None` if album is not a in a folder.
+
+**Note**: FolderInfo and AlbumInfo objects effectively work as a linked list.  The children of a folder are contained in `subfolders` and `album_info` and the parent object of both `AlbumInfo` and `FolderInfo` is represented by `parent`.  For example:
+
+```python
+>>> import osxphotos
+>>> photosdb = osxphotos.PhotosDB()
+>>> photosdb.folder_info
+[<osxphotos.albuminfo.FolderInfo object at 0x10fcc0160>]
+>>> photosdb.folder_info[0].title
+'Folder1'
+>>> photosdb.folder_info[0].subfolders[1].title
+'SubFolder2'
+>>> photosdb.folder_info[0].subfolders[1].album_info[0].title
+'AlbumInFolder'
+>>> photosdb.folder_info[0].subfolders[1].album_info[0].parent.title
+'SubFolder2'
+>>> photosdb.folder_info[0].subfolders[1].album_info[0].parent.album_info[0].title
+'AlbumInFolder'
+```
+
 
 ### PlaceInfo
 [PhotoInfo.place](#place) returns a PlaceInfo object if the photo contains valid reverse geolocation information.  PlaceInfo has the following properties.  
@@ -1172,13 +1306,20 @@ If you have an interesting example that shows usage of this package, submit an i
 
 Testing against "real world" Photos libraries would be especially helpful.  If you discover issues in testing against your Photos libraries, please open an issue.  I've done extensive testing against my own Photos library but that's a since data point and I'm certain there are issues lurking in various edge cases I haven't discovered yet.
 
+## Known Bugs
+
+My goal is make osxphotos as reliable and comprehensive as possible.  The test suite currently has over 400 tests--but there are still some [bugs](https://github.com/RhetTbull/osxphotos/issues?q=is%3Aissue+is%3Aopen+label%3Abug) or incomplete features lurking.  If you find bugs please open an [issue](https://github.com/RhetTbull/osxphotos/issues).  Notable issues include:
+
+- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos.  osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`.  If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored.  See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101) Note: Alpha version of fix for this bug is implemented in the current version of osxphotos.
+- The `--download-missing` option for `osxphotos export` does not work correctly with burst images.  It will download the primary image but not the other burst images.  See [Issue #75](https://github.com/RhetTbull/osxphotos/issues/75)
+
 ## Implementation Notes
 
-This packge works by creating a copy of the sqlite3 database that photos uses to store data about the photos library. the class photosdb then queries this database to extract information about the photos such as persons (faces identified in the photos), albums, keywords, etc.  If your library is large, the database can be hundreds of MB in size and the copy then read can take many 10s of seconds to complete.  Once copied, the entire database is processed and an in-memory data structure is created meaning all subsequent accesses of the PhotosDB object occur much more quickly. 
+This package works by creating a copy of the sqlite3 database that photos uses to store data about the photos library. The class PhotosDB then queries this database to extract information about the photos such as persons (faces identified in the photos), albums, keywords, etc.  If your library is large, the database can be hundreds of MB in size and the copy read then  can take many 10s of seconds to complete.  Once copied, the entire database is processed and an in-memory data structure is created meaning all subsequent accesses of the PhotosDB object occur much more quickly. 
 
 If apple changes the database format this will likely break.
 
-Apple does provide a framework ([PhotoKit](https://developer.apple.com/documentation/photokit?language=objc)) for querying the user's Photos library and I attempted to create the funcationality in this package using this framework but unfortunately PhotoKit does not provide access to much of the needed metadata (such as Faces/Persons).  While copying the sqlite file is a bit kludgy, it allows osxphotos to provide access to all available metadata.
+Apple does provide a framework ([PhotoKit](https://developer.apple.com/documentation/photokit?language=objc)) for querying the user's Photos library and I attempted to create the funcationality in this package using this framework but unfortunately PhotoKit does not provide access to much of the needed metadata (such as Faces/Persons) and Apple's System Integrity Protection (SIP) made the interface unreliable.  If you'd like to experiment with the PhotoKit interface, here's some sample [code](https://gist.github.com/RhetTbull/41cc85e5bdeb30f761147ce32fba5c94).  While copying the sqlite file is a bit kludgy, it allows osxphotos to provide access to all available metadata.
 
 ## Dependencies
 - [PyObjC](https://pythonhosted.org/pyobjc/)

examples/examples.py

@@ -14,7 +14,7 @@ def main():
 
     print(photosdb.keywords)
     print(photosdb.persons)
-    print(photosdb.album_names)
+    print(photosdb.albums)
 
     print(photosdb.keywords_as_dict)
     print(photosdb.persons_as_dict)

osxphotos/__main__.py

@@ -187,7 +187,7 @@ def query_options(f):
             metavar="KEYWORD",
             default=None,
             multiple=True,
-            help="Search for keyword KEYWORD. "
+            help="Search for photos with keyword KEYWORD. "
             'If more than one keyword, treated as "OR", e.g. find photos match any keyword',
         ),
         o(
@@ -195,7 +195,7 @@ def query_options(f):
             metavar="PERSON",
             default=None,
             multiple=True,
-            help="Search for person PERSON. "
+            help="Search for photos with person PERSON. "
             'If more than one person, treated as "OR", e.g. find photos match any person',
         ),
         o(
@@ -203,15 +203,24 @@ def query_options(f):
             metavar="ALBUM",
             default=None,
             multiple=True,
-            help="Search for album ALBUM. "
+            help="Search for photos in album ALBUM. "
             'If more than one album, treated as "OR", e.g. find photos match any album',
         ),
+        o(
+            "--folder",
+            metavar="FOLDER",
+            default=None,
+            multiple=True,
+            help="Search for photos in an album in folder FOLDER. "
+            'If more than one folder, treated as "OR", e.g. find photos in any FOLDER.  '
+            "Only searches top level folders (e.g. does not look at subfolders)",
+        ),
         o(
             "--uuid",
             metavar="UUID",
             default=None,
             multiple=True,
-            help="Search for UUID(s).",
+            help="Search for photos with UUID(s).",
         ),
         o(
             "--title",
@@ -336,6 +345,11 @@ def query_options(f):
             is_flag=True,
             help="Search for photos that are not panoramas.",
         ),
+        o(
+            "--has-raw",
+            is_flag=True,
+            help="Search for photos with both a jpeg and RAW version",
+        ),
         o(
             "--only-movies",
             is_flag=True,
@@ -670,6 +684,7 @@ def query(
     keyword,
     person,
     album,
+    folder,
     uuid,
     title,
     no_title,
@@ -714,6 +729,7 @@ def query(
     not_selfie,
     panorama,
     not_panorama,
+    has_raw,
     place,
     no_place,
 ):
@@ -728,10 +744,12 @@ def query(
         keyword,
         person,
         album,
+        folder,
         uuid,
         edited,
         external_edit,
         uti,
+        has_raw,
         from_date,
         to_date,
     ]
@@ -781,6 +799,7 @@ def query(
         keyword=keyword,
         person=person,
         album=album,
+        folder=folder,
         uuid=uuid,
         title=title,
         no_title=no_title,
@@ -824,6 +843,7 @@ def query(
         not_selfie=not_selfie,
         panorama=panorama,
         not_panorama=not_panorama,
+        has_raw=has_raw,
         place=place,
         no_place=no_place,
     )
@@ -835,8 +855,8 @@ def query(
 
 @cli.command(cls=ExportCommand)
 @DB_OPTION
-@query_options
 @click.option("--verbose", "-V", is_flag=True, help="Print verbose output.")
+@query_options
 @click.option(
     "--overwrite",
     is_flag=True,
@@ -852,27 +872,33 @@ def query(
     "(e.g. DEST/2019/12/20/photoname.jpg).",
 )
 @click.option(
-    "--export-edited",
+    "--skip-edited",
     is_flag=True,
-    help="Also export edited version of photo if an edited version exists.  "
-    'Edited photo will be named in form of "photoname_edited.ext"',
+    help="Do not export edited version of photo if an edited version exists.",
 )
 @click.option(
-    "--export-bursts",
+    "--skip-bursts",
     is_flag=True,
-    help="If a photo is a burst photo export all associated burst images in the library.  "
-    "Not currently compatible with --download-misssing; see note on --download-missing.",
+    help="Do not export all associated burst images in the library if a photo is a burst photo.  ",
 )
 @click.option(
-    "--export-live",
+    "--skip-live",
     is_flag=True,
-    help="If a photo is a live photo export the associated live video component."
-    "  Live video will have same name as photo but with .mov extension. ",
+    help="Do not export the associated live video component of a live photo.",
 )
 @click.option(
-    "--original-name",
+    "--skip-raw",
     is_flag=True,
-    help="Use photo's original filename instead of current filename for export.",
+    help="Do not export associated RAW images of a RAW/jpeg pair.  "
+    "Note: this does not skip RAW photos if the RAW photo does not have an associated jpeg image "
+    "(e.g. the RAW file was imported to Photos without a jpeg preview).",
+)
+@click.option(
+    "--current-name",
+    is_flag=True,
+    help="Use photo's current filename instead of original filename for export.  "
+    "Note: Starting with Photos 5, all photos are renamed upon import.  By default, "
+    "photos are exported with the the original name they had before import.",
 )
 @click.option(
     "--sidecar",
@@ -895,7 +921,7 @@ def query(
     "to interact with Photos to export the photo which will force Photos to download from iCloud if "
     "the photo does not exist on disk.  This will be slow and will require internet connection. "
     "This obviously only works if the Photos library is synched to iCloud.  "
-    "Note: --download-missing is not currently compatabile with --export-bursts; "
+    "Note: --download-missing does not currently export all burst images; "
     "only the primary photo will be exported--associated burst images will be skipped.",
 )
 @click.option(
@@ -932,6 +958,7 @@ def export(
     keyword,
     person,
     album,
+    folder,
     uuid,
     title,
     no_title,
@@ -952,10 +979,11 @@ def export(
     verbose,
     overwrite,
     export_by_date,
-    export_edited,
-    export_bursts,
-    export_live,
-    original_name,
+    skip_edited,
+    skip_bursts,
+    skip_live,
+    skip_raw,
+    current_name,
     sidecar,
     only_photos,
     only_movies,
@@ -980,6 +1008,7 @@ def export(
     not_selfie,
     panorama,
     not_panorama,
+    has_raw,
     directory,
     place,
     no_place,
@@ -991,6 +1020,10 @@ def export(
         if more than one option is provided, they are treated as "AND" 
         (e.g. search for photos matching all options).
         If no query options are provided, all photos will be exported.
+        By default, all versions of all photos will be exported including edited
+        versions, live photo movies, burst photos, and associated RAW images. 
+        See --skip-edited, --skip-live, --skip-bursts, and --skip-raw options
+        to modify this behavior. 
     """
 
     if not os.path.isdir(dest):
@@ -1019,6 +1052,17 @@ def export(
         click.echo(cli.commands["export"].get_help(ctx), err=True)
         return
 
+    # initialize export flags
+    # by default, will export all versions of photos unless skip flag is set
+    (export_edited, export_bursts, export_live, export_raw) = [
+        not x for x in [skip_edited, skip_bursts, skip_live, skip_raw]
+    ]
+
+    # though the command line option is current_name, internally all processing
+    # logic uses original_name which is the boolean inverse of current_name
+    # because the original code used --original-name as an option
+    original_name = not current_name
+
     # verify exiftool installed an in path
     if exiftool:
         try:
@@ -1051,6 +1095,7 @@ def export(
         keyword=keyword,
         person=person,
         album=album,
+        folder=folder,
         uuid=uuid,
         title=title,
         no_title=no_title,
@@ -1094,6 +1139,7 @@ def export(
         not_selfie=not_selfie,
         panorama=panorama,
         not_panorama=not_panorama,
+        has_raw=has_raw,
         place=place,
         no_place=no_place,
     )
@@ -1127,6 +1173,7 @@ def export(
                         exiftool,
                         directory,
                         no_extended_attributes,
+                        export_raw,
                     )
         else:
             for p in photos:
@@ -1144,6 +1191,7 @@ def export(
                     exiftool,
                     directory,
                     no_extended_attributes,
+                    export_raw,
                 )
                 if export_paths:
                     click.echo(f"Exported {p.filename} to {export_paths}")
@@ -1218,6 +1266,9 @@ def print_photo_info(photos, json=False):
                 "hdr",
                 "selfie",
                 "panorama",
+                "has_raw",
+                "uti_raw",
+                "path_raw",
             ]
         )
         for p in photos:
@@ -1259,6 +1310,9 @@ def print_photo_info(photos, json=False):
                     p.hdr,
                     p.selfie,
                     p.panorama,
+                    p.has_raw,
+                    p.uti_raw,
+                    p.path_raw,
                 ]
             )
         for row in dump:
@@ -1270,6 +1324,7 @@ def _query(
     keyword=None,
     person=None,
     album=None,
+    folder=None,
     uuid=None,
     title=None,
     no_title=None,
@@ -1313,13 +1368,14 @@ def _query(
     not_selfie=None,
     panorama=None,
     not_panorama=None,
+    has_raw=None,
     place=None,
     no_place=None,
 ):
-    """ run a query against PhotosDB to extract the photos based on user supply criteria """
-    """ used by query and export commands """
-    """ arguments must be passed in same order as query and export """
-    """ if either is modified, need to ensure all three functions are updated """
+    """ run a query against PhotosDB to extract the photos based on user supply criteria 
+        used by query and export commands 
+        arguments must be passed in same order as query and export 
+        if either is modified, need to ensure all three functions are updated """
 
     photosdb = osxphotos.PhotosDB(dbfile=db)
     photos = photosdb.photos(
@@ -1333,6 +1389,21 @@ def _query(
         to_date=to_date,
     )
 
+    if folder:
+        # search for photos in an album in folder
+        # finds photos that have albums whose top level folder matches folder
+        photo_list = []
+        for f in folder:
+            photo_list.extend(
+                [
+                    p
+                    for p in photos
+                    if p.album_info
+                    and f in [a.folder_names[0] for a in p.album_info if a.folder_names]
+                ]
+            )
+        photos = photo_list
+
     if title:
         # search title field for text
         # if more than one, find photos with all title values in title
@@ -1486,6 +1557,9 @@ def _query(
     elif not_incloud:
         photos = [p for p in photos if not p.incloud]
 
+    if has_raw:
+        photos = [p for p in photos if p.has_raw]
+
     return photos
 
 
@@ -1503,6 +1577,7 @@ def export_photo(
     exiftool,
     directory,
     no_extended_attributes,
+    export_raw,
 ):
     """ Helper function for export that does the actual export
         photo: PhotoInfo object
@@ -1518,6 +1593,7 @@ def export_photo(
         exiftool: use exiftool to write EXIF metadata directly to exported photo
         directory: template used to determine output directory
         no_extended_attributes: boolean; if True, exports photo without preserving extended attributes
+        export_raw: boolean; if True exports RAW image associate with the photo
         returns list of path(s) of exported photo or None if photo was missing 
     """
 
@@ -1596,6 +1672,7 @@ def export_photo(
             sidecar_json=sidecar_json,
             sidecar_xmp=sidecar_xmp,
             live_photo=export_live,
+            raw_photo=export_raw,
             overwrite=overwrite,
             use_photos_export=use_photos_export,
             exiftool=exiftool,

osxphotos/_constants.py

@@ -16,8 +16,9 @@ _TESTED_DB_VERSIONS = ["6000", "4025", "4016", "3301", "2622"]
 # only version 3 - 4 have RKVersion.selfPortrait
 _PHOTOS_3_VERSION = "3301"
 
-# versions later than this have a different database structure
-_PHOTOS_5_VERSION = "6000"
+# versions 5.0 and later have a different database structure
+_PHOTOS_4_VERSION = "4025"  # latest Mojove version on 10.14.6
+_PHOTOS_5_VERSION = "6000"  # seems to be current on 10.15.1 through 10.15.4
 
 # which major version operating systems have been tested
 _TESTED_OS_VERSIONS = ["12", "13", "14", "15"]
@@ -40,3 +41,13 @@ _MOVIE_TYPE = 1
 # Name of XMP template file
 _TEMPLATE_DIR = os.path.join(os.path.dirname(__file__), "templates")
 _XMP_TEMPLATE_NAME = "xmp_sidecar.mako"
+
+# Constants used for processing folders and albums
+_PHOTOS_5_ALBUM_KIND = 2  # normal user album
+_PHOTOS_5_SHARED_ALBUM_KIND = 1505  # shared album
+_PHOTOS_5_FOLDER_KIND = 4000  # user folder
+_PHOTOS_5_ROOT_FOLDER_KIND = 3999  # root folder
+
+_PHOTOS_4_ALBUM_KIND = 3  # RKAlbum.albumSubclass
+_PHOTOS_4_TOP_LEVEL_ALBUM = "TopLevelAlbums"
+_PHOTOS_4_ROOT_FOLDER = "LibraryFolder"

osxphotos/_version.py

@@ -1,3 +1,3 @@
 """ version info """
 
-__version__ = "0.26.1"
+__version__ = "0.28.2"

osxphotos/albuminfo.py

@@ -0,0 +1,211 @@
+"""
+AlbumInfo and FolderInfo classes for dealing with albums and folders
+
+AlbumInfo class
+Represents a single Album in the Photos library and provides access to the album's attributes
+PhotosDB.albums() returns a list of AlbumInfo objects
+
+FolderInfo class
+Represents a single Folder in the Photos library and provides access to the folders attributes
+PhotosDB.folders() returns a list of FolderInfo objects
+"""
+
+import logging
+
+from ._constants import (
+    _PHOTOS_4_ALBUM_KIND,
+    _PHOTOS_4_TOP_LEVEL_ALBUM,
+    _PHOTOS_4_VERSION,
+    _PHOTOS_5_ALBUM_KIND,
+    _PHOTOS_5_FOLDER_KIND,
+)
+
+
+class AlbumInfo:
+    """
+    Info about a specific Album, contains all the details about the album
+    including folders, photos, etc.
+    """
+
+    def __init__(self, db=None, uuid=None):
+        self._uuid = uuid
+        self._db = db
+        self._title = self._db._dbalbum_details[uuid]["title"]
+
+    @property
+    def title(self):
+        """ return title / name of album """
+        return self._title
+
+    @property
+    def uuid(self):
+        """ return uuid of album """
+        return self._uuid
+
+    @property
+    def photos(self):
+        """ return list of photos contained in album """
+        try:
+            return self._photos
+        except AttributeError:
+            uuid = self._db._dbalbums_album[self._uuid]
+            self._photos = self._db.photos(uuid=uuid)
+            return self._photos
+
+    @property
+    def folder_names(self):
+        """ return hierarchical list of folders the album is contained in
+            the folder list is in form:
+            ["Top level folder", "sub folder 1", "sub folder 2", ...]
+            returns empty list if album is not in any folders """
+
+        try:
+            return self._folder_names
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                self._folder_names = self._db._album_folder_hierarchy_list(self._uuid)
+            else:
+                self._folder_names = self._db._album_folder_hierarchy_list(self._uuid)
+            return self._folder_names
+
+    @property
+    def folder_list(self):
+        """ return hierarchical list of folders the album is contained in
+            as list of FolderInfo objects in form 
+            ["Top level folder", "sub folder 1", "sub folder 2", ...]
+            returns empty list if album is not in any folders """
+
+        try:
+            return self._folders
+        except AttributeError:
+            self._folders = self._db._album_folder_hierarchy_folderinfo(self._uuid)
+            return self._folders
+
+    @property
+    def parent(self):
+        """ returns FolderInfo object for parent folder or None if no parent (e.g. top-level album) """
+        try:
+            return self._parent
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                parent_uuid = self._db._dbalbum_details[self._uuid]["folderUuid"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=parent_uuid)
+                    if parent_uuid != _PHOTOS_4_TOP_LEVEL_ALBUM
+                    else None
+                )
+            else:
+                parent_pk = self._db._dbalbum_details[self._uuid]["parentfolder"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=self._db._dbalbums_pk[parent_pk])
+                    if parent_pk != self._db._folder_root_pk
+                    else None
+                )
+            return self._parent
+
+    def __len__(self):
+        """ return number of photos contained in album """
+        return len(self.photos)
+
+
+class FolderInfo:
+    """
+    Info about a specific folder, contains all the details about the folder 
+    including folders, albums, etc
+    """
+
+    def __init__(self, db=None, uuid=None):
+        self._uuid = uuid
+        self._db = db
+        if self._db._db_version <= _PHOTOS_4_VERSION:
+            self._pk = None
+            self._title = self._db._dbfolder_details[uuid]["name"]
+        else:
+            self._pk = self._db._dbalbum_details[uuid]["pk"]
+            self._title = self._db._dbalbum_details[uuid]["title"]
+
+    @property
+    def title(self):
+        """ return title / name of folder"""
+        return self._title
+
+    @property
+    def uuid(self):
+        """ return uuid of folder """
+        return self._uuid
+
+    @property
+    def album_info(self):
+        """ return list of albums (as AlbumInfo objects) contained in the folder """
+        try:
+            return self._albums
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                albums = [
+                    AlbumInfo(db=self._db, uuid=album)
+                    for album, detail in self._db._dbalbum_details.items()
+                    if not detail["intrash"]
+                    and detail["albumSubclass"] == _PHOTOS_4_ALBUM_KIND
+                    and detail["folderUuid"] == self._uuid
+                ]
+            else:
+                albums = [
+                    AlbumInfo(db=self._db, uuid=album)
+                    for album, detail in self._db._dbalbum_details.items()
+                    if not detail["intrash"]
+                    and detail["kind"] == _PHOTOS_5_ALBUM_KIND
+                    and detail["parentfolder"] == self._pk
+                ]
+            self._albums = albums
+            return self._albums
+
+    @property
+    def parent(self):
+        """ returns FolderInfo object for parent or None if no parent (e.g. top-level folder) """
+        try:
+            return self._parent
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                parent_uuid = self._db._dbfolder_details[self._uuid]["parentFolderUuid"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=parent_uuid)
+                    if parent_uuid != _PHOTOS_4_TOP_LEVEL_ALBUM
+                    else None
+                )
+            else:
+                parent_pk = self._db._dbalbum_details[self._uuid]["parentfolder"]
+                self._parent = (
+                    FolderInfo(db=self._db, uuid=self._db._dbalbums_pk[parent_pk])
+                    if parent_pk != self._db._folder_root_pk
+                    else None
+                )
+            return self._parent
+
+    @property
+    def subfolders(self):
+        """ return list of folders (as FolderInfo objects) contained in the folder """
+        try:
+            return self._folders
+        except AttributeError:
+            if self._db._db_version <= _PHOTOS_4_VERSION:
+                folders = [
+                    FolderInfo(db=self._db, uuid=folder)
+                    for folder, detail in self._db._dbfolder_details.items()
+                    if not detail["intrash"]
+                    and not detail["isMagic"]
+                    and detail["parentFolderUuid"] == self._uuid
+                ]
+            else:
+                folders = [
+                    FolderInfo(db=self._db, uuid=album)
+                    for album, detail in self._db._dbalbum_details.items()
+                    if not detail["intrash"]
+                    and detail["kind"] == _PHOTOS_5_FOLDER_KIND
+                    and detail["parentfolder"] == self._pk
+                ]
+            self._folders = folders
+            return self._folders
+
+    def __len__(self):
+        """ returns count of folders + albums contained in the folder """
+        return len(self.subfolders) + len(self.album_info)

osxphotos/photoinfo.py

@@ -21,18 +21,21 @@ from mako.template import Template
 from ._constants import (
     _MOVIE_TYPE,
     _PHOTO_TYPE,
+    _PHOTOS_4_VERSION,
     _PHOTOS_5_SHARED_PHOTO_PATH,
-    _PHOTOS_5_VERSION,
     _TEMPLATE_DIR,
     _XMP_TEMPLATE_NAME,
 )
 from .exiftool import ExifTool
 from .placeinfo import PlaceInfo4, PlaceInfo5
+from .albuminfo import AlbumInfo
 from .utils import (
     _copy_file,
     _export_photo_uuid_applescript,
     _get_resource_loc,
     dd_to_dms_str,
+    findfiles,
+    get_preferred_uti_extension,
 )
 
 
@@ -95,7 +98,7 @@ class PhotoInfo:
         if self._info["isMissing"] == 1:
             return photopath  # path would be meaningless until downloaded
 
-        if self._db._db_version < _PHOTOS_5_VERSION:
+        if self._db._db_version <= _PHOTOS_4_VERSION:
             vol = self._info["volume"]
             if vol is not None:
                 photopath = os.path.join("/Volumes", vol, self._info["imagePath"])
@@ -134,7 +137,7 @@ class PhotoInfo:
 
         photopath = None
 
-        if self._db._db_version < _PHOTOS_5_VERSION:
+        if self._db._db_version <= _PHOTOS_4_VERSION:
             if self._info["hasAdjustments"]:
                 edit_id = self._info["edit_resource_id"]
                 if edit_id is not None:
@@ -238,6 +241,75 @@ class PhotoInfo:
 
         return photopath
 
+    @property
+    def path_raw(self):
+        """ absolute path of associated RAW image or None if there is not one """
+
+        # In Photos 5, raw is in same folder as original but with _4.ext
+        # Unless "Copy Items to the Photos Library" is not checked
+        # then RAW image is not renamed but has same name is jpeg buth with raw extension
+        # Current implementation uses findfiles to find images with the correct raw UTI extension
+        # in same folder as the original and with same stem as original in form: original_stem*.raw_ext
+        # TODO: I don't like this -- would prefer a more deterministic approach but until I have more
+        # data on how Photos stores and retrieves RAW images, this seems to be working
+
+        if self._info["isMissing"] == 1:
+            return None  # path would be meaningless until downloaded
+
+        if not self.has_raw:
+            return None  # no raw image to get path for
+
+        # if self._info["shared"]:
+        #     # shared photo
+        #     photopath = os.path.join(
+        #         self._db._library_path,
+        #         _PHOTOS_5_SHARED_PHOTO_PATH,
+        #         self._info["directory"],
+        #         self._info["filename"],
+        #     )
+        #     return photopath
+
+        if self._db._db_version <= _PHOTOS_4_VERSION:
+            vol = self._info["raw_info"]["volume"]
+            if vol is not None:
+                photopath = os.path.join(
+                    "/Volumes", vol, self._info["raw_info"]["imagePath"]
+                )
+            else:
+                photopath = os.path.join(
+                    self._db._masters_path, self._info["raw_info"]["imagePath"]
+                )
+            if not os.path.isfile(photopath):
+                logging.debug(
+                    f"MISSING PATH: RAW photo for UUID {self._uuid} should be at {photopath} but does not appear to exist"
+                )
+                photopath = None
+        else:
+            filestem = pathlib.Path(self._info["filename"]).stem
+            raw_ext = get_preferred_uti_extension(self._info["UTI_raw"])
+
+            if self._info["directory"].startswith("/"):
+                filepath = self._info["directory"]
+            else:
+                filepath = os.path.join(self._db._masters_path, self._info["directory"])
+
+            glob_str = f"{filestem}*.{raw_ext}"
+            raw_file = findfiles(glob_str, filepath)
+            if len(raw_file) != 1:
+                logging.warning(
+                    f"Error getting path to RAW file: {filepath}/{glob_str}"
+                )
+                photopath = None
+            else:
+                photopath = os.path.join(filepath, raw_file[0])
+                if not os.path.isfile(photopath):
+                    logging.debug(
+                        f"MISSING PATH: RAW photo for UUID {self._uuid} should be at {photopath} but does not appear to exist"
+                    )
+                    photopath = None
+
+        return photopath
+
     @property
     def description(self):
         """ long / extended description of picture """
@@ -253,7 +325,18 @@ class PhotoInfo:
         """ list of albums picture is contained in """
         albums = []
         for album in self._info["albums"]:
-            albums.append(self._db._dbalbum_details[album]["title"])
+            if not self._db._dbalbum_details[album]["intrash"]:
+                albums.append(self._db._dbalbum_details[album]["title"])
+        return albums
+
+    @property
+    def album_info(self):
+        """ list of AlbumInfo objects representing albums the photos is contained in """
+        albums = []
+        for album in self._info["albums"]:
+            if not self._db._dbalbum_details[album]["intrash"]:
+                albums.append(AlbumInfo(db=self._db, uuid=album))
+
         return albums
 
     @property
@@ -317,7 +400,7 @@ class PhotoInfo:
     def shared(self):
         """ returns True if photos is in a shared iCloud album otherwise false
             Only valid on Photos 5; returns None on older versions """
-        if self._db._db_version >= _PHOTOS_5_VERSION:
+        if self._db._db_version > _PHOTOS_4_VERSION:
             return self._info["shared"]
         else:
             return None
@@ -329,6 +412,14 @@ class PhotoInfo:
         """
         return self._info["UTI"]
 
+    @property
+    def uti_raw(self):
+        """ Returns Uniform Type Identifier (UTI) for the RAW image if there is one
+            for example: com.canon.cr2-raw-image
+            Returns None if no associated RAW image
+        """
+        return self._info["UTI_raw"]
+
     @property
     def ismovie(self):
         """ Returns True if file is a movie, otherwise False
@@ -354,7 +445,7 @@ class PhotoInfo:
         """ Returns True if photo is a cloud asset (in an iCloud library),
             otherwise False 
         """
-        if self._db._db_version < _PHOTOS_5_VERSION:
+        if self._db._db_version <= _PHOTOS_4_VERSION:
             return (
                 True
                 if self._info["cloudLibraryState"] is not None
@@ -397,7 +488,7 @@ class PhotoInfo:
             If photo is missing, returns None """
 
         photopath = None
-        if self._db._db_version < _PHOTOS_5_VERSION:
+        if self._db._db_version <= _PHOTOS_4_VERSION:
             if self.live_photo and not self.ismissing:
                 live_model_id = self._info["live_model_id"]
                 if live_model_id == None:
@@ -488,7 +579,7 @@ class PhotoInfo:
         # implementation note: doesn't create the PlaceInfo object until requested
         # then memoizes the object in self._place to avoid recreating the object
 
-        if self._db._db_version < _PHOTOS_5_VERSION:
+        if self._db._db_version <= _PHOTOS_4_VERSION:
             try:
                 return self._place  # pylint: disable=access-member-before-definition
             except AttributeError:
@@ -509,12 +600,25 @@ class PhotoInfo:
                     self._place = None
                 return self._place
 
+    @property
+    def has_raw(self):
+        """ returns True if photo has an associated RAW image, otherwise False """
+        return self._info["has_raw"]
+
+    @property
+    def raw_original(self):
+        """ returns True if associated RAW image and the RAW image is selected in Photos
+            via "Use RAW as Original "
+            otherwise returns False """
+        return True if self._info["original_resource_choice"] == 1 else False
+
     def export(
         self,
         dest,
         *filename,
         edited=False,
         live_photo=False,
+        raw_photo=False,
         overwrite=False,
         increment=True,
         sidecar_json=False,
@@ -536,6 +640,7 @@ class PhotoInfo:
             edited: (boolean, default=False); if True will export the edited version of the photo 
                     (or raise exception if no edited version) 
             live_photo: (boolean, default=False); if True, will also export the associted .mov for live photos
+            raw_photo: (boolean, default=False); if True, will also export the associted RAW photo
             overwrite: (boolean, default=False); if True will overwrite files if they alreay exist 
             increment: (boolean, default=True); if True, will increment file name until a non-existant name is found 
                        if overwrite=False and increment=False, export will fail if destination file already exists 
@@ -612,7 +717,10 @@ class PhotoInfo:
         # warn if suffixes don't match but ignore .JPG / .jpeg as
         # Photo's often converts .JPG to .jpeg
         suffixes = sorted([x.lower() for x in [dest.suffix, actual_suffix]])
-        if dest.suffix != actual_suffix and suffixes != [".jpeg", ".jpg"]:
+        if dest.suffix.lower() != actual_suffix.lower() and suffixes != [
+            ".jpeg",
+            ".jpg",
+        ]:
             logging.warning(
                 f"Invalid destination suffix: {dest.suffix}, should be {actual_suffix}"
             )
@@ -685,6 +793,20 @@ class PhotoInfo:
                     exported_files.append(str(live_name))
                 else:
                     logging.warning(f"Skipping missing live movie for {filename}")
+
+            # copy associated RAW image if requested
+            if raw_photo and self.has_raw:
+                raw_path = pathlib.Path(self.path_raw)
+                raw_ext = raw_path.suffix
+                raw_name = dest.parent / f"{dest.stem}{raw_ext}"
+                if raw_path is not None:
+                    logging.debug(
+                        f"Exporting RAW photo of {filename} as {raw_name.name}"
+                    )
+                    _copy_file(str(raw_path), str(raw_name), norsrc=no_xattr)
+                    exported_files.append(str(raw_name))
+                else:
+                    logging.warning(f"Skipping missing RAW photo for {filename}")
         else:
             # use_photo_export
             exported = None
@@ -935,6 +1057,9 @@ class PhotoInfo:
             "hdr": self.hdr,
             "selfie": self.selfie,
             "panorama": self.panorama,
+            "has_raw": self.has_raw,
+            "uti_raw": self.uti_raw,
+            "path_raw": self.path_raw,
         }
         return yaml.dump(info, sort_keys=False)
 
@@ -981,6 +1106,9 @@ class PhotoInfo:
             "hdr": self.hdr,
             "selfie": self.selfie,
             "panorama": self.panorama,
+            "has_raw": self.has_raw,
+            "uti_raw": self.uti_raw,
+            "path_raw": self.path_raw,
         }
         return json.dumps(pic)
 

osxphotos/template.py

@@ -10,6 +10,7 @@
 # This code isn't elegant but it seems to work well.  PRs gladly accepted.
 
 import datetime
+import os
 import pathlib
 import re
 from typing import Tuple, List  # pylint: disable=syntax-error
@@ -55,6 +56,7 @@ TEMPLATE_SUBSTITUTIONS = {
 # Permitted multi-value substitutions (each of these returns None or 1 or more values)
 TEMPLATE_SUBSTITUTIONS_MULTI_VALUED = {
     "{album}": "Album(s) photo is contained in",
+    "{folder_album}": "Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder",
     "{keyword}": "Keyword(s) assigned to photo",
     "{person}": "Person(s) / face(s) in a photo",
 }
@@ -321,6 +323,19 @@ def render_filepath_template(template, photo, none_str="_"):
             values = photo.persons
             # remove any _UNKNOWN_PERSON values
             values = [val for val in values if val != _UNKNOWN_PERSON]
+        elif field == "folder_album":
+            values = []
+            # photos must be in an album to be in a folder
+            for album in photo.album_info:
+                if album.folder_names:
+                    # album in folder
+                    folder = os.path.sep.join(album.folder_names)
+                    folder += os.path.sep + album.title
+                    values.append(folder)
+                else:
+                    # album not in folder
+                    values.append(album.title)
+
         else:
             raise ValueError(f"Unhandleded template value: {field}")
 

osxphotos/utils.py

@@ -1,8 +1,11 @@
+import fnmatch
 import glob
 import logging
+import os
 import os.path
 import pathlib
 import platform
+import re
 import sqlite3
 import subprocess
 import sys
@@ -11,6 +14,7 @@ import urllib.parse
 from plistlib import load as plistload
 
 import CoreFoundation
+import CoreServices
 import objc
 from Foundation import *
 
@@ -302,6 +306,28 @@ def create_path_by_date(dest, dt):
     return new_dest
 
 
+def get_preferred_uti_extension(uti):
+    """ get preferred extension for a UTI type
+        uti: UTI str, e.g. 'public.jpeg'
+        returns: preferred extension as str """
+
+    # reference: https://developer.apple.com/documentation/coreservices/1442744-uttypecopypreferredtagwithclass?language=objc
+
+    ext = CoreServices.UTTypeCopyPreferredTagWithClass(
+        uti, CoreServices.kUTTagClassFilenameExtension
+    )
+    return ext
+
+
+def findfiles(pattern, path_):
+    """Returns list of filenames from path_ matched by pattern
+       shell pattern. Matching is case-insensitive."""
+    # See: https://gist.github.com/techtonik/5694830
+
+    rule = re.compile(fnmatch.translate(pattern), re.IGNORECASE)
+    return [name for name in os.listdir(path_) if rule.match(name)]
+
+
 # TODO: this doesn't always work, still looking for a way to
 # force Photos to open the library being operated on
 # def _open_photos_library_applescript(library_path):

tests/Test-10.12.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2019-12-08T16:44:38Z</date>
+		<date>2020-04-17T18:39:50Z</date>
 	</dict>
 	<key>PXPeopleScreenUnlocked</key>
 	<true/>

tests/Test-10.12.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-01-22T02:10:26Z</date>
+	<date>2020-04-17T18:40:46Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-01-22T02:10:27Z</date>
+	<date>2020-04-17T18:39:51Z</date>
 </dict>
 </plist>

tests/Test-10.12.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2020-01-19T17:29:28Z</date>
+	<date>2020-04-17T18:39:52Z</date>
 </dict>
 </plist>

tests/Test-10.12.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>414</integer>
+	<integer>502</integer>
 	<key>LibraryBuildTag</key>
 	<string>E3E46F2A-7168-4973-AB3E-5848F80BFC7D</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.12.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>414</integer>
+		<integer>502</integer>
 		<key>LibraryBuildTag</key>
 		<string>E3E46F2A-7168-4973-AB3E-5848F80BFC7D</string>
 		<key>LibrarySchemaVersion</key>

tests/Test-10.14.6.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -9,12 +9,14 @@
 		<key>ExpandedSidebarItemIdentifiers</key>
 		<array>
 			<string>TopLevelAlbums</string>
+			<string>QtSnVvTkQ%i2z3hB834M1A</string>
 			<string>TopLevelSlideshows</string>
+			<string>N7eQ4VhfTfeHFp9PPHaJDw</string>
 		</array>
 		<key>IPXWorkspaceControllerZoomLevelsKey</key>
 		<dict>
 			<key>kZoomLevelIdentifierAlbums</key>
-			<integer>10</integer>
+			<integer>5</integer>
 			<key>kZoomLevelIdentifierVersions</key>
 			<integer>7</integer>
 		</dict>
@@ -23,11 +25,11 @@
 			<key>key</key>
 			<integer>1</integer>
 			<key>lastKnownDisplayName</key>
-			<string>Test Album (1)</string>
+			<string>Pumpkin Farm (1)</string>
 			<key>type</key>
 			<string>album</string>
 			<key>uuid</key>
-			<string>Uq6qsKihRRSjMHTiD+0Azg</string>
+			<string>xJ8ya3NBRWC24gKhcwwNeQ</string>
 		</dict>
 		<key>lastKnownItemCounts</key>
 		<dict>

tests/Test-10.14.6.photoslibrary/private/com.apple.photoanalysisd/GraphService/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-03-27T04:00:09Z</date>
+	<date>2020-04-18T18:01:02Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-03-27T04:00:10Z</date>
+	<date>2020-04-18T17:22:55Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/private/com.apple.photomodel/appPrivateData.plist

@@ -5,7 +5,7 @@
 	<key>LithiumMessageTracer</key>
 	<dict>
 		<key>LastReportedDate</key>
-		<date>2020-03-15T20:19:24Z</date>
+		<date>2020-04-17T17:51:16Z</date>
 	</dict>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/analysismetadata.plist

@@ -11,6 +11,6 @@
 	<key>PLLastRevGeoForcedProviderOutOfDateCheckVersionKey</key>
 	<integer>1</integer>
 	<key>PLLastRevGeoVerFileFetchDateKey</key>
-	<date>2020-03-27T03:59:54Z</date>
+	<date>2020-04-17T17:49:52Z</date>
 </dict>
 </plist>

tests/Test-10.14.6.photoslibrary/resources/moments/historicalmarker.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>LastHistoryRowId</key>
-	<integer>575</integer>
+	<integer>606</integer>
 	<key>LibraryBuildTag</key>
 	<string>D8C4AAA1-3AB6-4A65-BEBD-99CC3E5D433E</string>
 	<key>LibrarySchemaVersion</key>

tests/Test-10.14.6.photoslibrary/resources/recovery/Info.plist

@@ -9,7 +9,7 @@
 	<key>HistoricalMarker</key>
 	<dict>
 		<key>LastHistoryRowId</key>
-		<integer>575</integer>
+		<integer>606</integer>
 		<key>LibraryBuildTag</key>
 		<string>D8C4AAA1-3AB6-4A65-BEBD-99CC3E5D433E</string>
 		<key>LibrarySchemaVersion</key>
@@ -24,7 +24,7 @@
 	<key>SnapshotCompletedDate</key>
 	<date>2019-07-27T13:16:43Z</date>
 	<key>SnapshotLastValidated</key>
-	<date>2020-03-27T04:02:59Z</date>
+	<date>2020-04-17T17:51:16Z</date>
 	<key>SnapshotTables</key>
 	<dict/>
 </dict>

tests/Test-10.15.1.photoslibrary/database/Photos.sqlite.lock

@@ -7,7 +7,7 @@
 	<key>hostuuid</key>
 	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
 	<key>pid</key>
-	<integer>685</integer>
+	<integer>900</integer>
 	<key>processname</key>
 	<string>photolibraryd</string>
 	<key>uid</key>

tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist

@@ -3,24 +3,24 @@
 <plist version="1.0">
 <dict>
 	<key>BackgroundHighlightCollection</key>
-	<date>2020-04-11T01:34:25Z</date>
+	<date>2020-04-17T14:33:32Z</date>
 	<key>BackgroundHighlightEnrichment</key>
-	<date>2020-04-11T01:34:24Z</date>
+	<date>2020-04-17T14:33:32Z</date>
 	<key>BackgroundJobAssetRevGeocode</key>
-	<date>2020-04-11T01:34:26Z</date>
+	<date>2020-04-17T14:33:33Z</date>
 	<key>BackgroundJobSearch</key>
-	<date>2020-04-11T01:34:26Z</date>
+	<date>2020-04-17T14:33:33Z</date>
 	<key>BackgroundPeopleSuggestion</key>
-	<date>2020-04-11T01:34:23Z</date>
+	<date>2020-04-17T14:33:31Z</date>
 	<key>BackgroundUserBehaviorProcessor</key>
-	<date>2020-04-10T06:04:05Z</date>
+	<date>2020-04-17T07:32:04Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundGraphConsistencyUpdateJobDateKey</key>
-	<date>2020-04-11T01:34:33Z</date>
+	<date>2020-04-17T14:33:37Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate</key>
-	<date>2020-04-10T06:04:04Z</date>
+	<date>2020-04-17T07:32:00Z</date>
 	<key>PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate</key>
-	<date>2020-04-11T01:34:26Z</date>
+	<date>2020-04-17T14:33:34Z</date>
 	<key>SiriPortraitDonation</key>
-	<date>2020-04-10T06:04:05Z</date>
+	<date>2020-04-17T07:32:04Z</date>
 </dict>
 </plist>

tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist

@@ -3,8 +3,8 @@
 <plist version="1.0">
 <dict>
 	<key>FaceIDModelLastGenerationKey</key>
-	<date>2020-04-10T06:04:06Z</date>
+	<date>2020-04-17T07:32:07Z</date>
 	<key>LastContactClassificationKey</key>
-	<date>2020-04-10T06:04:08Z</date>
+	<date>2020-04-17T07:32:12Z</date>
 </dict>
 </plist>

tests/Test-10.15.1.photoslibrary/resources/journals/Album.plist

@@ -3,7 +3,7 @@
 <plist version="1.0">
 <dict>
 	<key>coalesceDate</key>
-	<date>2019-12-08T18:06:37Z</date>
+	<date>2020-04-11T16:34:16Z</date>
 	<key>coalescePayloadVersion</key>
 	<integer>1</integer>
 	<key>currentPayloadVersion</key>

tests/Test-10.15.4.photoslibrary/database/Photos.sqlite.lock

@@ -7,7 +7,7 @@
 	<key>hostuuid</key>
 	<string>9575E48B-8D5F-5654-ABAC-4431B1167324</string>
 	<key>pid</key>
-	<integer>1440</integer>
+	<integer>3212</integer>
 	<key>processname</key>
 	<string>photolibraryd</string>
 	<key>uid</key>

tests/Test-10.15.4.photoslibrary/private/com.apple.Photos/appPrivateData.plist

@@ -6,7 +6,9 @@
 	<array/>
 	<key>ExpandedSidebarItemIdentifiers</key>
 	<array>
+		<string>88A5F8B8-5B9A-43C7-BB85-3952B81580EB/L0/020</string>
 		<string>92D68107-B6C7-453B-96D2-97B0F26D5B8B/L0/020</string>
+		<string>29EF7A97-7E76-4D5F-A5E0-CC0A93E8524C/L0/020</string>
 	</array>
 	<key>Photos</key>
 	<dict>