diff --git a/README.md b/README.md index e72ba16f..4d3001c1 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ + [AlbumInfo](#albuminfo) + [FolderInfo](#folderinfo) + [PlaceInfo](#placeinfo) - + [Template Functions](#template-functions) + + [Template Substitutions](#template-substitutions) + [Utility Functions](#utility-functions) * [Examples](#examples) * [Related Projects](#related-projects) @@ -101,6 +101,7 @@ Example: `osxphotos help export` ``` Usage: osxphotos export [OPTIONS] [PHOTOS_LIBRARY]... DEST +Usage: __main__.py export [OPTIONS] [PHOTOS_LIBRARY]... DEST Export photos from the Photos database. Export path DEST is required. Optionally, query the Photos database using 1 or more search options; if @@ -223,6 +224,18 @@ Options: exporting metadata. --album-keyword Use album name as keyword/tag when exporting metadata. + --keyword-template TEMPLATE For use with --exiftool, --sidecar; specify + a template string to use as keyword in the + form '{name,DEFAULT}' This is the same + format as --directory. For example, if you + wanted to add the full path to the folder + and album photo is contained in as a keyword + when exporting you could specify --keyword- + template "{folder_album}" You may specify + more than one template, for example + --keyword-template "{folder_album}" + --keyword-template "{created.year}" See + Templating System below. --current-name Use photo's current filename instead of original filename for export. Note: Starting with Photos 5, all photos are @@ -270,7 +283,7 @@ Options: **Templating System** -With the --directory option, you may specify a template for the export +With the --directory option you may specify a template for the export directory. This directory will be appended to the export path specified in the export DEST argument to export. For example, if template is '{created.year}/{created.month}', and export desitnation DEST is @@ -278,6 +291,11 @@ the export DEST argument to export. For example, if template is be '/Users/maria/Pictures/export/2020/March' if the photo was created in March 2020. +The templating system may also be used with the --keyword-template option to +set keywords on export (with --exiftool or --sidecar), for example, to set a +new keyword in format 'folder/subfolder/album' to preserve the folder/album +structure, you can use --keyword-template "{folder_album}" + In the template, valid template substitutions will be replaced by the corresponding value from the table below. Invalid substitutions will result in a an error and the script will abort. @@ -302,7 +320,7 @@ I plan to eventually extend the templating system to the exported filename so you can specify the filename using a template. Substitution Description -{name} Filename of the photo +{name} Current filename of the photo {original_name} Photo's original filename when imported to Photos {title} Title of the photo @@ -1004,6 +1022,26 @@ If overwrite=False and increment=False, export will fail if destination file alr **Implementation Note**: Because the usual python file copy methods don't preserve all the metadata available on MacOS, export uses `/usr/bin/ditto` to do the copy for export. ditto preserves most metadata such as extended attributes, permissions, ACLs, etc. +#### `render_template()` + +`render_template(template, none_str = "_", path_sep = None)` +Render template string for photo. none_str is used if template substitution results in None value and no default specified. +- `template`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None. This is useful for values which are not always present, for example reverse geolocation data. +- `none_str`: optional str to use as substitution when template value is None and no default specified in the template string. default is "_". +- `path_sep`: optional character to use as path separator, default is os.path.sep + +Returns a tuple of (rendered, unmatched) where rendered is a list of rendered strings with all substitutions made and unmatched is a list of any strings that resembled a template substitution but did not match a known substitution. E.g. if template contained "{foo}", unmatched would be ["foo"]. + +e.g. `render_filepath_template("{created.year}/{foo}", photo)` would return `(["2020/{foo}"],["foo"])` + +If you want to include "{" or "}" in the output, use "{{" or "}}" + +e.g. `render_filepath_template("{created.year}/{{foo}}", photo)` would return `(["2020/{foo}"],[])` + +Some substitutions, notably `album`, `keyword`, and `person` could return multiple values, hence a new string will be return for each possible substitution (hence why a list of rendered strings is returned). For example, a photo in 2 albums: 'Vacation' and 'Family' would result in the following rendered values if template was "{created.year}/{album}" and created.year == 2020: `["2020/Vacation","2020/Family"]` + +See [Template Substitutions](#template-substitutions) for additional details. + ### AlbumInfo PhotosDB.album_info and PhotoInfo.album_info return a list of AlbumInfo objects. Each AlbumInfo object represents a single album in the Photos library. @@ -1149,26 +1187,9 @@ PostalAddress(street='3700 Wailea Alanui Dr', sub_locality=None, city='Kihei', s '96753' ``` -### Template Functions - -There is a simple template system used by the command line client to specify the output directory using a template. The following are available in `osxphotos.template`. - -#### `render_filepath_template(template, photo, none_str="_")` -Render template string for photo. none_str is used if template substitution results in None value and no default specified. -- `template`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None. This is useful for values which are not always present, for example reverse geolocation data. -- `photo`: a [PhotoInfo](#photoinfo) object -- `none_str`: optional str to use as substitution when template value is None and no default specified in the template string. default is "_". - -Returns a tuple of (rendered, unmatched) where rendered is a list of rendered strings with all substitutions made and unmatched is a list of any strings that resembled a template substitution but did not match a known substitution. E.g. if template contained "{foo}", unmatched would be ["foo"]. - -e.g. `render_filepath_template("{created.year}/{foo}", photo)` would return `(["2020/{foo}"],["foo"])` - -If you want to include "{" or "}" in the output, use "{{" or "}}" - -e.g. `render_filepath_template("{created.year}/{{foo}}", photo)` would return `(["2020/{foo}"],[])` - -Some substitutions, notably `album`, `keyword`, and `person` could return multiple values, hence a new string will be return for each possible substitution (hence why a list of rendered strings is returned). For example, a photo in 2 albums: 'Vacation' and 'Family' would result in the following rendered values if template was "{created.year}/{album}" and created.year == 2020: `["2020/Vacation","2020/Family"]` +### Template Substitutions +The following substitutions are availabe for use with `PhotoInfo.render_template()` | Substitution | Description | |--------------|-------------| @@ -1208,21 +1229,6 @@ Some substitutions, notably `album`, `keyword`, and `person` could return multip |{person}|Person(s) / face(s) in a photo| -#### `DateTimeFormatter(dt)` -Class that provides easy access to formatted datetime values. -- `dt`: a datetime.datetime object - -Returnes `DateTimeFormater` class. - -Has the following properties: -- `date`: Date in ISO format without timezone, e.g. "2020-03-04" -- `year`: 4-digit year -- `yy`: 2-digit year -- `month`: month name in user's locale -- `mon`: month abbreviation in user's locale -- `mm`: 2-digit month -- `doy`: 3-digit day of year (e.g. Julian day) - ### Utility Functions The following functions are located in osxphotos.utils @@ -1328,7 +1334,7 @@ Testing against "real world" Photos libraries would be especially helpful. If y My goal is make osxphotos as reliable and comprehensive as possible. The test suite currently has over 400 tests--but there are still some [bugs](https://github.com/RhetTbull/osxphotos/issues?q=is%3Aissue+is%3Aopen+label%3Abug) or incomplete features lurking. If you find bugs please open an [issue](https://github.com/RhetTbull/osxphotos/issues). Notable issues include: -- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos. osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`. If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored. See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101) Note: Alpha version of fix for this bug is implemented in the current version of osxphotos. +- RAW images imported to Photos with an associated jpeg preview are not handled correctly by osxphotos. osxphotos query and export will operate on the jpeg preview instead of the RAW image as will `PhotoInfo.path`. If the user selects "Use RAW as original" in Photos, the RAW image will be exported or operated on but the jpeg will be ignored. See [Issue #101](https://github.com/RhetTbull/osxphotos/issues/101) Note: Beta version of fix for this bug is implemented in the current version of osxphotos. - The `--download-missing` option for `osxphotos export` does not work correctly with burst images. It will download the primary image but not the other burst images. See [Issue #75](https://github.com/RhetTbull/osxphotos/issues/75) ## Implementation Notes diff --git a/osxphotos/__main__.py b/osxphotos/__main__.py index e56dae5f..d8c0cdae 100644 --- a/osxphotos/__main__.py +++ b/osxphotos/__main__.py @@ -21,11 +21,7 @@ import osxphotos from ._constants import _EXIF_TOOL_URL, _PHOTOS_4_VERSION, _UNKNOWN_PLACE from ._version import __version__ from .exiftool import get_exiftool_path -from .template import ( - render_filepath_template, - TEMPLATE_SUBSTITUTIONS, - TEMPLATE_SUBSTITUTIONS_MULTI_VALUED, -) +from .template import TEMPLATE_SUBSTITUTIONS, TEMPLATE_SUBSTITUTIONS_MULTI_VALUED from .utils import _copy_file, create_path_by_date @@ -83,7 +79,7 @@ class ExportCommand(click.Command): formatter.write_text("**Templating System**") formatter.write("\n") formatter.write_text( - "With the --directory option, you may specify a template for the " + "With the --directory option you may specify a template for the " + "export directory. This directory will be appended to the export path specified " + "in the export DEST argument to export. For example, if template is " + "'{created.year}/{created.month}', and export desitnation DEST is " @@ -92,6 +88,13 @@ class ExportCommand(click.Command): + "if the photo was created in March 2020. " ) formatter.write("\n") + formatter.write_text( + "The templating system may also be used with the --keyword-template option " + + "to set keywords on export (with --exiftool or --sidecar), " + + "for example, to set a new keyword in format 'folder/subfolder/album' to " + + 'preserve the folder/album structure, you can use --keyword-template "{folder_album}"' + ) + formatter.write("\n") formatter.write_text( "In the template, valid template substitutions will be replaced by " + "the corresponding value from the table below. Invalid substitutions will result in a " @@ -903,6 +906,20 @@ def query( is_flag=True, help="Use album name as keyword/tag when exporting metadata.", ) +@click.option( + "--keyword-template", + metavar="TEMPLATE", + multiple=True, + default=None, + help="For use with --exiftool, --sidecar; specify a template string to use as " + "keyword in the form '{name,DEFAULT}' " + "This is the same format as --directory. For example, if you wanted to add " + "the full path to the folder and album photo is contained in as a keyword when exporting " + 'you could specify --keyword-template "{folder_album}" ' + 'You may specify more than one template, for example --keyword-template "{folder_album}" ' + '--keyword-template "{created.year}" ' + "See Templating System below.", +) @click.option( "--current-name", is_flag=True, @@ -995,6 +1012,7 @@ def export( skip_raw, person_keyword, album_keyword, + keyword_template, current_name, sidecar, only_photos, @@ -1188,6 +1206,7 @@ def export( export_raw, album_keyword, person_keyword, + keyword_template, ) else: for p in photos: @@ -1208,6 +1227,7 @@ def export( export_raw, album_keyword, person_keyword, + keyword_template, ) if export_paths: click.echo(f"Exported {p.filename} to {export_paths}") @@ -1596,6 +1616,7 @@ def export_photo( export_raw, album_keyword, person_keyword, + keyword_template, ): """ Helper function for export that does the actual export photo: PhotoInfo object @@ -1614,6 +1635,7 @@ def export_photo( export_raw: boolean; if True exports RAW image associate with the photo album_keyword: boolean; if True, exports album names as keywords in metadata person_keyword: boolean; if True, exports person names as keywords in metadata + keyword_template: list of strings; if provided use rendered template strings as keywords returns list of path(s) of exported photo or None if photo was missing """ @@ -1654,7 +1676,7 @@ def export_photo( dest_paths = [dest_path] elif directory: # got a directory template, render it and check results are valid - dirnames, unmatched = render_filepath_template(directory, photo) + dirnames, unmatched = photo.render_template(directory) if unmatched: raise click.BadOptionUsage( "directory", @@ -1699,6 +1721,7 @@ def export_photo( no_xattr=no_extended_attributes, use_albums_as_keywords=album_keyword, use_persons_as_keywords=person_keyword, + keyword_template=keyword_template, )[0] photo_paths.append(photo_path) @@ -1736,6 +1759,7 @@ def export_photo( no_xattr=no_extended_attributes, use_albums_as_keywords=album_keyword, use_persons_as_keywords=person_keyword, + keyword_template=keyword_template, ) return photo_paths diff --git a/osxphotos/_constants.py b/osxphotos/_constants.py index 714a32aa..cfdeeddb 100644 --- a/osxphotos/_constants.py +++ b/osxphotos/_constants.py @@ -51,3 +51,12 @@ _PHOTOS_5_ROOT_FOLDER_KIND = 3999 # root folder _PHOTOS_4_ALBUM_KIND = 3 # RKAlbum.albumSubclass _PHOTOS_4_TOP_LEVEL_ALBUM = "TopLevelAlbums" _PHOTOS_4_ROOT_FOLDER = "LibraryFolder" + +# EXIF related constants +# max keyword length for IPTC:Keyword, reference +# https://www.iptc.org/std/photometadata/documentation/userguide/ +_MAX_IPTC_KEYWORD_LEN = 64 + +# Sentinel value for detecting if a template in keyword_template doesn't match +# If anyone has a keyword matching this, then too bad... +_OSXPHOTOS_NONE_SENTINEL = "OSXPhotosXYZZY42_Sentinel$" diff --git a/osxphotos/_version.py b/osxphotos/_version.py index b7040edb..9815fa64 100644 --- a/osxphotos/_version.py +++ b/osxphotos/_version.py @@ -1,3 +1,3 @@ """ version info """ -__version__ = "0.28.11" +__version__ = "0.28.13" diff --git a/osxphotos/datetime_formatter.py b/osxphotos/datetime_formatter.py new file mode 100644 index 00000000..d5fdeef3 --- /dev/null +++ b/osxphotos/datetime_formatter.py @@ -0,0 +1,52 @@ +""" Simple formatting of datetime.datetime objects """ + +import datetime + + +class DateTimeFormatter: + """ provides property access to formatted datetime.datetime strftime values """ + + def __init__(self, dt: datetime.datetime): + self.dt = dt + + @property + def date(self): + """ ISO date in form 2020-03-22 """ + date = self.dt.date().isoformat() + return date + + @property + def year(self): + """ 4 digit year """ + year = f"{self.dt.year}" + return year + + @property + def yy(self): + """ 2 digit year """ + yy = f"{self.dt.strftime('%y')}" + return yy + + @property + def mm(self): + """ 2 digit month """ + mm = f"{self.dt.strftime('%m')}" + return mm + + @property + def month(self): + """ Month as locale's full name """ + month = f"{self.dt.strftime('%B')}" + return month + + @property + def mon(self): + """ Month as locale's abbreviated name """ + mon = f"{self.dt.strftime('%b')}" + return mon + + @property + def doy(self): + """ Julian day of year starting from 001 """ + doy = f"{self.dt.strftime('%j')}" + return doy diff --git a/osxphotos/photoinfo.py b/osxphotos/photoinfo.py index 00fd6e22..1df32457 100644 --- a/osxphotos/photoinfo.py +++ b/osxphotos/photoinfo.py @@ -7,6 +7,7 @@ PhotosDB.photos() returns a list of PhotoInfo objects import glob import json import logging +import os import os.path import pathlib import re @@ -19,7 +20,9 @@ import yaml from mako.template import Template from ._constants import ( + _MAX_IPTC_KEYWORD_LEN, _MOVIE_TYPE, + _OSXPHOTOS_NONE_SENTINEL, _PHOTO_TYPE, _PHOTOS_4_VERSION, _PHOTOS_5_SHARED_PHOTO_PATH, @@ -27,9 +30,15 @@ from ._constants import ( _UNKNOWN_PERSON, _XMP_TEMPLATE_NAME, ) +from .albuminfo import AlbumInfo +from .datetime_formatter import DateTimeFormatter from .exiftool import ExifTool from .placeinfo import PlaceInfo4, PlaceInfo5 -from .albuminfo import AlbumInfo +from .template import ( + MULTI_VALUE_SUBSTITUTIONS, + TEMPLATE_SUBSTITUTIONS, + TEMPLATE_SUBSTITUTIONS_MULTI_VALUED, +) from .utils import ( _copy_file, _export_photo_uuid_applescript, @@ -636,6 +645,7 @@ class PhotoInfo: no_xattr=False, use_albums_as_keywords=False, use_persons_as_keywords=False, + keyword_template=None, ): """ export photo dest: must be valid destination path (or exception raised) @@ -666,6 +676,7 @@ class PhotoInfo: when exporting metadata with exiftool or sidecar use_persons_as_keywords: (boolean, default = False); if True, will include person names in keywords when exporting metadata with exiftool or sidecar + keyword_template: (list of strings); list of template strings that will be rendered as used as keywords """ # list of all files exported during this call to export @@ -873,6 +884,7 @@ class PhotoInfo: sidecar_str = self._exiftool_json_sidecar( use_albums_as_keywords=use_albums_as_keywords, use_persons_as_keywords=use_persons_as_keywords, + keyword_template=keyword_template, ) try: self._write_sidecar(sidecar_filename, sidecar_str) @@ -886,6 +898,7 @@ class PhotoInfo: sidecar_str = self._xmp_sidecar( use_albums_as_keywords=use_albums_as_keywords, use_persons_as_keywords=use_persons_as_keywords, + keyword_template=keyword_template, ) try: self._write_sidecar(sidecar_filename, sidecar_str) @@ -900,12 +913,350 @@ class PhotoInfo: exported_file, use_albums_as_keywords=use_albums_as_keywords, use_persons_as_keywords=use_persons_as_keywords, + keyword_template=keyword_template, ) return exported_files + def render_template(self, template, none_str="_", path_sep=None): + """ render a filename or directory template + template: str template + none_str: str to use default for None values, default is '_' + path_sep: optional character to use as path separator, default is os.path.sep """ + + if path_sep is None: + path_sep = os.path.sep + elif path_sep is not None and len(path_sep) != 1: + raise ValueError(f"path_sep must be single character: {path_sep}") + + # the rendering happens in two phases: + # phase 1: handle all the single-value template substitutions + # results in a single string with all the template fields replaced + # phase 2: loop through all the multi-value template substitutions + # could result in multiple strings + # e.g. if template is "{album}/{person}" and there are 2 albums and 3 persons in the photo + # there would be 6 possible renderings (2 albums x 3 persons) + + # regex to find {template_field,optional_default} in strings + # for explanation of regex see https://regex101.com/r/4JJg42/1 + # pylint: disable=anomalous-backslash-in-string + regex = r"(? _MAX_IPTC_KEYWORD_LEN + ] + if long_keywords: + logging.warning( + f"Some keywords exceed max IPTC Keyword length of {_MAX_IPTC_KEYWORD_LEN}: {long_keywords}" + ) + + logging.debug(f"rendered_keywords: {rendered_keywords}") + keyword_list.extend(rendered_keywords) + if keyword_list: exif["XMP:TagsList"] = exif["IPTC:Keywords"] = keyword_list @@ -982,6 +1373,7 @@ class PhotoInfo: if self.keywords or person_list: # Photos puts both keywords and persons in Subject when using "Export IPTC as XMP" + # only use Photos' keywords for subject exif["XMP:Subject"] = list(self.keywords) + person_list # if self.favorite(): @@ -1016,8 +1408,17 @@ class PhotoInfo: json_str = json.dumps([exif]) return json_str - def _xmp_sidecar(self, use_albums_as_keywords=False, use_persons_as_keywords=False): - """ returns string for XMP sidecar """ + def _xmp_sidecar( + self, + use_albums_as_keywords=False, + use_persons_as_keywords=False, + keyword_template=None, + ): + """ returns string for XMP sidecar + use_albums_as_keywords: treat album names as keywords + use_persons_as_keywords: treat person names as keywords + keyword_template: (list of strings); list of template strings to render as keywords """ + # TODO: add additional fields to XMP file? xmp_template = Template( @@ -1028,6 +1429,9 @@ class PhotoInfo: if self.keywords: keyword_list.extend(self.keywords) + # TODO: keyword handling in this and _exiftool_json_sidecar is + # good candidate for pulling out in a function + person_list = [] if self.persons: # filter out _UNKNOWN_PERSON @@ -1039,6 +1443,39 @@ class PhotoInfo: if use_albums_as_keywords and self.albums: keyword_list.extend(self.albums) + if keyword_template: + rendered_keywords = [] + for template_str in keyword_template: + rendered, unmatched = self.render_template( + template_str, none_str=_OSXPHOTOS_NONE_SENTINEL, path_sep="/" + ) + if unmatched: + logging.warning( + f"Unmatched template substitution for template: {template_str} {unmatched}" + ) + rendered_keywords.extend(rendered) + + # filter out any template values that didn't match by looking for sentinel + rendered_keywords = [ + keyword + for keyword in rendered_keywords + if _OSXPHOTOS_NONE_SENTINEL not in keyword + ] + + # check to see if any keywords too long + long_keywords = [ + long_str + for long_str in rendered_keywords + if len(long_str) > _MAX_IPTC_KEYWORD_LEN + ] + if long_keywords: + logging.warning( + f"Some keywords exceed max IPTC Keyword length of {_MAX_IPTC_KEYWORD_LEN}: {long_keywords}" + ) + + logging.debug(f"rendered_keywords: {rendered_keywords}") + keyword_list.extend(rendered_keywords) + subject_list = [] if self.keywords or person_list: # Photos puts both keywords and persons in Subject when using "Export IPTC as XMP" diff --git a/osxphotos/template.py b/osxphotos/template.py index 551181db..90e802f6 100644 --- a/osxphotos/template.py +++ b/osxphotos/template.py @@ -1,4 +1,4 @@ -""" Custom template system for osxphotos """ +""" Custom template system for osxphotos (implemented in PhotoInfo.render_template) """ # Rolled my own template system because: # 1. Needed to handle multiple values (e.g. album, keyword) @@ -9,22 +9,14 @@ # # This code isn't elegant but it seems to work well. PRs gladly accepted. -import datetime import locale -import os -import pathlib -import re -from typing import Tuple, List # pylint: disable=syntax-error - -from .photoinfo import PhotoInfo -from ._constants import _UNKNOWN_PERSON # ensure locale set to user's locale locale.setlocale(locale.LC_ALL, "") # Permitted substitutions (each of these returns a single value or None) TEMPLATE_SUBSTITUTIONS = { - "{name}": "Filename of the photo", + "{name}": "Current filename of the photo", "{original_name}": "Photo's original filename when imported to Photos", "{title}": "Title of the photo", "{descr}": "Description of the photo", @@ -70,374 +62,3 @@ MULTI_VALUE_SUBSTITUTIONS = [ field.replace("{", "").replace("}", "") for field in TEMPLATE_SUBSTITUTIONS_MULTI_VALUED.keys() ] - - -def get_template_value(lookup, photo): - """ lookup template value (single-value template substitutions) for use in make_subst_function - lookup: value to find a match for - photo: PhotoInfo object whose data will be used for value substitutions - returns: either the matching template value (which may be None) - raises: KeyError if no rule exists for lookup """ - - # must be a valid keyword - if lookup == "name": - return pathlib.Path(photo.filename).stem - - if lookup == "original_name": - return pathlib.Path(photo.original_filename).stem - - if lookup == "title": - return photo.title - - if lookup == "descr": - return photo.description - - if lookup == "created.date": - return DateTimeFormatter(photo.date).date - - if lookup == "created.year": - return DateTimeFormatter(photo.date).year - - if lookup == "created.yy": - return DateTimeFormatter(photo.date).yy - - if lookup == "created.mm": - return DateTimeFormatter(photo.date).mm - - if lookup == "created.month": - return DateTimeFormatter(photo.date).month - - if lookup == "created.mon": - return DateTimeFormatter(photo.date).mon - - if lookup == "created.doy": - return DateTimeFormatter(photo.date).doy - - if lookup == "modified.date": - return ( - DateTimeFormatter(photo.date_modified).date if photo.date_modified else None - ) - - if lookup == "modified.year": - return ( - DateTimeFormatter(photo.date_modified).year if photo.date_modified else None - ) - - if lookup == "modified.yy": - return ( - DateTimeFormatter(photo.date_modified).yy if photo.date_modified else None - ) - - if lookup == "modified.mm": - return ( - DateTimeFormatter(photo.date_modified).mm if photo.date_modified else None - ) - - if lookup == "modified.month": - return ( - DateTimeFormatter(photo.date_modified).month - if photo.date_modified - else None - ) - - if lookup == "modified.mon": - return ( - DateTimeFormatter(photo.date_modified).mon if photo.date_modified else None - ) - - if lookup == "modified.doy": - return ( - DateTimeFormatter(photo.date_modified).doy if photo.date_modified else None - ) - - if lookup == "place.name": - return photo.place.name if photo.place else None - - if lookup == "place.country_code": - return photo.place.country_code if photo.place else None - - if lookup == "place.name.country": - return ( - photo.place.names.country[0] - if photo.place and photo.place.names.country - else None - ) - - if lookup == "place.name.state_province": - return ( - photo.place.names.state_province[0] - if photo.place and photo.place.names.state_province - else None - ) - - if lookup == "place.name.city": - return ( - photo.place.names.city[0] - if photo.place and photo.place.names.city - else None - ) - - if lookup == "place.name.area_of_interest": - return ( - photo.place.names.area_of_interest[0] - if photo.place and photo.place.names.area_of_interest - else None - ) - - if lookup == "place.address": - return ( - photo.place.address_str if photo.place and photo.place.address_str else None - ) - - if lookup == "place.address.street": - return ( - photo.place.address.street - if photo.place and photo.place.address.street - else None - ) - - if lookup == "place.address.city": - return ( - photo.place.address.city - if photo.place and photo.place.address.city - else None - ) - - if lookup == "place.address.state_province": - return ( - photo.place.address.state_province - if photo.place and photo.place.address.state_province - else None - ) - - if lookup == "place.address.postal_code": - return ( - photo.place.address.postal_code - if photo.place and photo.place.address.postal_code - else None - ) - - if lookup == "place.address.country": - return ( - photo.place.address.country - if photo.place and photo.place.address.country - else None - ) - - if lookup == "place.address.country_code": - return ( - photo.place.address.iso_country_code - if photo.place and photo.place.address.iso_country_code - else None - ) - - # if here, didn't get a match - raise KeyError(f"No rule for processing {lookup}") - - -def render_filepath_template(template, photo, none_str="_"): - """ render a filename or directory template - template: str template - photo: PhotoInfo object - none_str: str to use default for None values, default is '_' """ - - # the rendering happens in two phases: - # phase 1: handle all the single-value template substitutions - # results in a single string with all the template fields replaced - # phase 2: loop through all the multi-value template substitutions - # could result in multiple strings - # e.g. if template is "{album}/{person}" and there are 2 albums and 3 persons in the photo - # there would be 6 possible renderings (2 albums x 3 persons) - - # regex to find {template_field,optional_default} in strings - # for explanation of regex see https://regex101.com/r/4JJg42/1 - # pylint: disable=anomalous-backslash-in-string - regex = r"(?hostuuid 9575E48B-8D5F-5654-ABAC-4431B1167324 pid - 2322 + 725 processname photolibraryd uid diff --git a/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite b/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite index d2cd81c8..0f72cdaf 100644 Binary files a/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite and b/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite differ diff --git a/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite-shm b/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite-shm index fe9ac284..3d5d2ccc 100644 Binary files a/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite-shm and b/tests/Test-10.15.1.photoslibrary/database/search/psi.sqlite-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db b/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db index 57dc31e8..258a10e9 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db and b/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db-shm index 34ce768b..facab9c5 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-shm index 1c22cdab..f2945c36 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-wal b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-wal index f691e94b..4e511313 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-wal and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSBusinessCategoryCache.POI.sqlite-wal differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-shm index 9caf9859..427f22cd 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-wal b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-wal index e0aaa005..3892cbff 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-wal and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSContactCache.sqlite-wal differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-shm index 23a3792e..c8d13ffc 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-wal b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-wal index 56dd73f7..48a9311e 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-wal and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/CLSLocationCache.sqlite-wal differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-shm index 06b48e30..4b6b8ba3 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-wal b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-wal index 14f6afff..ee4b7e98 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-wal and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PGCurationCache.sqlite.sqlite-wal differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist index 932681cb..b1d90f7c 100644 --- a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist +++ b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotoAnalysisServicePreferences.plist @@ -3,24 +3,24 @@ BackgroundHighlightCollection - 2020-04-29T06:08:11Z + 2020-05-01T23:03:12Z BackgroundHighlightEnrichment - 2020-04-29T06:08:11Z + 2020-05-01T23:03:11Z BackgroundJobAssetRevGeocode - 2020-04-29T06:08:11Z + 2020-05-02T01:35:19Z BackgroundJobSearch - 2020-04-29T06:08:11Z + 2020-05-01T23:03:12Z BackgroundPeopleSuggestion - 2020-04-29T06:08:11Z + 2020-05-01T23:03:11Z BackgroundUserBehaviorProcessor - 2020-04-29T06:08:11Z + 2020-05-01T23:03:13Z PhotoAnalysisGraphLastBackgroundGraphConsistencyUpdateJobDateKey - 2020-04-29T06:08:13Z + 2020-05-02T01:35:36Z PhotoAnalysisGraphLastBackgroundGraphRebuildJobDate - 2020-04-29T06:08:10Z + 2020-05-01T23:03:11Z PhotoAnalysisGraphLastBackgroundMemoryGenerationJobDate - 2020-04-29T06:08:12Z + 2020-05-02T01:35:19Z SiriPortraitDonation - 2020-04-29T06:08:11Z + 2020-05-01T23:03:13Z diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/construction-photosgraph.kgdb-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/construction-photosgraph.kgdb-shm index ca5a3e01..fe9ac284 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/construction-photosgraph.kgdb-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/construction-photosgraph.kgdb-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph-tmp.kgdb-shm b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph-tmp.kgdb-shm index ca5a3e01..fe9ac284 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph-tmp.kgdb-shm and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph-tmp.kgdb-shm differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph.kgdb b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph.kgdb index aba6bab7..2c32774f 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph.kgdb and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/graph/PhotosGraph/photosgraph.kgdb differ diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist index eac29459..e6dee62e 100644 --- a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist +++ b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/PhotoAnalysisServicePreferences.plist @@ -3,8 +3,8 @@ FaceIDModelLastGenerationKey - 2020-04-29T06:08:12Z + 2020-05-01T23:03:14Z LastContactClassificationKey - 2020-04-29T06:08:14Z + 2020-05-01T23:03:18Z diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/faceWorkerState.plist b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/faceWorkerState.plist index c36206de..736db69d 100644 --- a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/faceWorkerState.plist +++ b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/faceWorkerState.plist @@ -3,7 +3,7 @@ IncrementalPersonProcessingStage - 6 + 0 PersonBuilderLastMinimumFaceGroupSizeForCreatingMergeCandidates 15 PersonBuilderMergeCandidatesEnabled diff --git a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/vnpersonsmodel.bin b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/vnpersonsmodel.bin index c4d8bbee..c3a6c0e5 100644 Binary files a/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/vnpersonsmodel.bin and b/tests/Test-10.15.1.photoslibrary/private/com.apple.photoanalysisd/caches/vision/vnpersonsmodel.bin differ diff --git a/tests/test_cli.py b/tests/test_cli.py index f533e497..b106855a 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -332,6 +332,7 @@ def test_export_sidecar(): "-V", ], ) + assert result.exit_code == 0 files = glob.glob("*.*") assert sorted(files) == sorted(CLI_EXPORT_SIDECAR_FILENAMES) @@ -855,3 +856,70 @@ def test_no_folder_1_14(): json_got = json.loads(result.output) assert len(json_got) == 1 # single element assert json_got[0]["uuid"] == "15uNd7%8RguTEgNPKHfTWw" + + +def test_export_sidecar_keyword_template(): + import json + import glob + import os + import os.path + import osxphotos + + from osxphotos.__main__ import cli + + runner = CliRunner() + cwd = os.getcwd() + # pylint: disable=not-context-manager + with runner.isolated_filesystem(): + result = runner.invoke( + cli, + [ + "export", + "--db", + os.path.join(cwd, CLI_PHOTOS_DB), + ".", + "--sidecar=json", + "--sidecar=xmp", + "--keyword-template", + "{folder_album}", + f"--uuid={CLI_EXPORT_UUID}", + "-V", + ], + ) + assert result.exit_code == 0 + files = glob.glob("*.*") + assert sorted(files) == sorted(CLI_EXPORT_SIDECAR_FILENAMES) + + json_expected = json.loads( + """ + [{"_CreatedBy": "osxphotos, https://github.com/RhetTbull/osxphotos", + "EXIF:ImageDescription": "Girl holding pumpkin", + "XMP:Description": "Girl holding pumpkin", + "XMP:Title": "I found one!", + "XMP:TagsList": ["Kids", "Multi Keyword", "Test Album", "Pumpkin Farm"], + "IPTC:Keywords": ["Kids", "Multi Keyword", "Test Album", "Pumpkin Farm"], + "XMP:PersonInImage": ["Katie"], + "XMP:Subject": ["Kids", "Katie"], + "EXIF:DateTimeOriginal": "2018:09:28 16:07:07", + "EXIF:OffsetTimeOriginal": "-04:00", + "EXIF:ModifyDate": "2020:04:11 12:34:16"}]""" + )[0] + + import logging + + json_file = open("Pumkins2.json", "r") + json_got = json.load(json_file)[0] + json_file.close() + + # some gymnastics to account for different sort order in different pythons + for k, v in json_got.items(): + if type(v) in (list, tuple): + assert sorted(json_expected[k]) == sorted(v) + else: + assert json_expected[k] == v + + for k, v in json_expected.items(): + if type(v) in (list, tuple): + assert sorted(json_got[k]) == sorted(v) + else: + assert json_got[k] == v diff --git a/tests/test_export_catalina_10_15_1.py b/tests/test_export_catalina_10_15_1.py index 0847a186..25e4477c 100644 --- a/tests/test_export_catalina_10_15_1.py +++ b/tests/test_export_catalina_10_15_1.py @@ -467,7 +467,6 @@ def test_exiftool_json_sidecar(): json_got = photos[0]._exiftool_json_sidecar() json_got = json.loads(json_got)[0] - # some gymnastics to account for different sort order in different pythons # some gymnastics to account for different sort order in different pythons for k, v in json_got.items(): if type(v) in (list, tuple): @@ -508,7 +507,6 @@ def test_exiftool_json_sidecar_use_persons_keyword(): json_got = photos[0]._exiftool_json_sidecar(use_persons_as_keywords=True) json_got = json.loads(json_got)[0] - # some gymnastics to account for different sort order in different pythons # some gymnastics to account for different sort order in different pythons for k, v in json_got.items(): if type(v) in (list, tuple): @@ -549,7 +547,6 @@ def test_exiftool_json_sidecar_use_albums_keyword(): json_got = photos[0]._exiftool_json_sidecar(use_albums_as_keywords=True) json_got = json.loads(json_got)[0] - # some gymnastics to account for different sort order in different pythons # some gymnastics to account for different sort order in different pythons for k, v in json_got.items(): if type(v) in (list, tuple): @@ -612,14 +609,16 @@ def test_xmp_sidecar(): 2018-09-28T15:35:49 - """ + """ xmp_expected_lines = [line.strip() for line in xmp_expected.split("\n")] xmp_got = photos[0]._xmp_sidecar() xmp_got_lines = [line.strip() for line in xmp_got.split("\n")] - for line_expected, line_got in zip(xmp_expected_lines, xmp_got_lines): + for line_expected, line_got in zip( + sorted(xmp_expected_lines), sorted(xmp_got_lines) + ): assert line_expected == line_got @@ -673,14 +672,16 @@ def test_xmp_sidecar_use_persons_keyword(): 2018-09-28T15:35:49 - """ + """ xmp_expected_lines = [line.strip() for line in xmp_expected.split("\n")] xmp_got = photos[0]._xmp_sidecar(use_persons_as_keywords=True) xmp_got_lines = [line.strip() for line in xmp_got.split("\n")] - for line_expected, line_got in zip(xmp_expected_lines, xmp_got_lines): + for line_expected, line_got in zip( + sorted(xmp_expected_lines), sorted(xmp_got_lines) + ): assert line_expected == line_got @@ -734,12 +735,14 @@ def test_xmp_sidecar_use_albums_keyword(): 2018-09-28T15:35:49 - """ + """ xmp_expected_lines = [line.strip() for line in xmp_expected.split("\n")] xmp_got = photos[0]._xmp_sidecar(use_albums_as_keywords=True) xmp_got_lines = [line.strip() for line in xmp_got.split("\n")] - for line_expected, line_got in zip(xmp_expected_lines, xmp_got_lines): + for line_expected, line_got in zip( + sorted(xmp_expected_lines), sorted(xmp_got_lines) + ): assert line_expected == line_got diff --git a/tests/test_export_catalina_10_15_1_use_photos_export.py b/tests/test_export_catalina_10_15_1_use_photos_export.py index 1750a2f2..87d9d1b5 100644 --- a/tests/test_export_catalina_10_15_1_use_photos_export.py +++ b/tests/test_export_catalina_10_15_1_use_photos_export.py @@ -155,4 +155,3 @@ def test_export_edited_no_edit(photosdb): with pytest.raises(Exception) as e: assert photos[0].export(dest, use_photos_export=True, edited=True) assert e.type == ValueError - diff --git a/tests/test_export_keyword_template_catalina_10_15_4.py b/tests/test_export_keyword_template_catalina_10_15_4.py new file mode 100644 index 00000000..9d7633ec --- /dev/null +++ b/tests/test_export_keyword_template_catalina_10_15_4.py @@ -0,0 +1,219 @@ +import pytest + +from osxphotos._constants import _UNKNOWN_PERSON + +PHOTOS_DB = "./tests/Test-10.15.4.photoslibrary/database/photos.db" + +TOP_LEVEL_FOLDERS = ["Folder1"] + +TOP_LEVEL_CHILDREN = ["SubFolder1", "SubFolder2"] + +FOLDER_ALBUM_DICT = {"Folder1": [], "SubFolder1": [], "SubFolder2": ["AlbumInFolder"]} + +ALBUM_NAMES = ["Pumpkin Farm", "AlbumInFolder", "Test Album", "Test Album"] + +ALBUM_PARENT_DICT = { + "Pumpkin Farm": None, + "AlbumInFolder": "SubFolder2", + "Test Album": None, +} + +ALBUM_FOLDER_NAMES_DICT = { + "Pumpkin Farm": [], + "AlbumInFolder": ["Folder1", "SubFolder2"], + "Test Album": [], +} + +ALBUM_LEN_DICT = {"Pumpkin Farm": 3, "AlbumInFolder": 2, "Test Album": 1} + +ALBUM_PHOTO_UUID_DICT = { + "Pumpkin Farm": [ + "F12384F6-CD17-4151-ACBA-AE0E3688539E", + "D79B8D77-BFFC-460B-9312-034F2877D35B", + "1EB2B765-0765-43BA-A90C-0D0580E6172C", + ], + "Test Album": [ + "F12384F6-CD17-4151-ACBA-AE0E3688539E", + "D79B8D77-BFFC-460B-9312-034F2877D35B", + ], + "AlbumInFolder": [ + "3DD2C897-F19E-4CA6-8C22-B027D5A71907", + "E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51", + ], +} + +UUID_DICT = { + "two_albums": "F12384F6-CD17-4151-ACBA-AE0E3688539E", + "in_album": "E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51", + "xmp": "F12384F6-CD17-4151-ACBA-AE0E3688539E", +} + + +def test_exiftool_json_sidecar_keyword_template_long(caplog): + import osxphotos + from osxphotos._constants import _MAX_IPTC_KEYWORD_LEN + import json + + photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB) + photos = photosdb.photos(uuid=[UUID_DICT["in_album"]]) + + json_expected = json.loads( + """ + [{"_CreatedBy": "osxphotos, https://github.com/RhetTbull/osxphotos", + "EXIF:ImageDescription": "Bride Wedding day", + "XMP:Description": "Bride Wedding day", + "XMP:TagsList": ["wedding", "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"], + "IPTC:Keywords": ["wedding", "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"], + "XMP:PersonInImage": ["Maria"], + "XMP:Subject": ["wedding", "Maria"], + "EXIF:DateTimeOriginal": "2019:04:15 14:40:24", + "EXIF:OffsetTimeOriginal": "-04:00", "EXIF:ModifyDate": "2019:11:24 13:09:17"}] + """ + )[0] + + long_str = "x" * (_MAX_IPTC_KEYWORD_LEN + 1) + json_got = photos[0]._exiftool_json_sidecar(keyword_template=[long_str]) + json_got = json.loads(json_got)[0] + + assert "Some keywords exceed max IPTC Keyword length" in caplog.text + # some gymnastics to account for different sort order in different pythons + for k, v in json_got.items(): + if type(v) in (list, tuple): + assert sorted(json_expected[k]) == sorted(v) + else: + assert json_expected[k] == v + + for k, v in json_expected.items(): + if type(v) in (list, tuple): + assert sorted(json_got[k]) == sorted(v) + else: + assert json_got[k] == v + + for k, v in json_expected.items(): + if type(v) in (list, tuple): + assert sorted(json_got[k]) == sorted(v) + else: + assert json_got[k] == v + + +def test_exiftool_json_sidecar_keyword_template(): + import osxphotos + import json + + photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB) + photos = photosdb.photos(uuid=[UUID_DICT["in_album"]]) + + json_expected = json.loads( + """ + [{"_CreatedBy": "osxphotos, https://github.com/RhetTbull/osxphotos", + "EXIF:ImageDescription": "Bride Wedding day", + "XMP:Description": "Bride Wedding day", + "XMP:TagsList": ["wedding", "Folder1/SubFolder2/AlbumInFolder"], + "IPTC:Keywords": ["wedding", "Folder1/SubFolder2/AlbumInFolder"], + "XMP:PersonInImage": ["Maria"], + "XMP:Subject": ["wedding", "Maria"], + "EXIF:DateTimeOriginal": "2019:04:15 14:40:24", + "EXIF:OffsetTimeOriginal": "-04:00", "EXIF:ModifyDate": "2019:11:24 13:09:17"}] + """ + )[0] + + json_got = photos[0]._exiftool_json_sidecar(keyword_template=["{folder_album}"]) + json_got = json.loads(json_got)[0] + + # some gymnastics to account for different sort order in different pythons + for k, v in json_got.items(): + if type(v) in (list, tuple): + assert sorted(json_expected[k]) == sorted(v) + else: + assert json_expected[k] == v + + for k, v in json_expected.items(): + if type(v) in (list, tuple): + assert sorted(json_got[k]) == sorted(v) + else: + assert json_got[k] == v + + # some gymnastics to account for different sort order in different pythons + for k, v in json_got.items(): + if type(v) in (list, tuple): + assert sorted(json_expected[k]) == sorted(v) + else: + assert json_expected[k] == v + + for k, v in json_expected.items(): + if type(v) in (list, tuple): + assert sorted(json_got[k]) == sorted(v) + else: + assert json_got[k] == v + + for k, v in json_expected.items(): + if type(v) in (list, tuple): + assert sorted(json_got[k]) == sorted(v) + else: + assert json_got[k] == v + + +def test_xmp_sidecar_keyword_template(): + import osxphotos + + photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB) + photos = photosdb.photos(uuid=[UUID_DICT["xmp"]]) + + xmp_expected = """ + + + + + Girls with pumpkins + Can we carry this? + + + + Kids + Suzy + Katie + + + 2018-09-28T15:35:49.063000-04:00 + + + + + Suzy + Katie + + + + + + + Kids + Pumpkin Farm + Test Album + 2018 + + + + + 2018-09-28T15:35:49 + 2018-09-28T15:35:49 + + + """ + + xmp_expected_lines = [line.strip() for line in xmp_expected.split("\n")] + + xmp_got = photos[0]._xmp_sidecar( + keyword_template=["{created.year}", "{folder_album}"] + ) + xmp_got_lines = [line.strip() for line in xmp_got.split("\n")] + + for line_expected, line_got in zip( + sorted(xmp_expected_lines), sorted(xmp_got_lines) + ): + assert line_expected == line_got diff --git a/tests/test_export_mojave_10_14_6.py b/tests/test_export_mojave_10_14_6.py index 97e383e2..ee9810f3 100644 --- a/tests/test_export_mojave_10_14_6.py +++ b/tests/test_export_mojave_10_14_6.py @@ -454,7 +454,7 @@ def test_xmp_sidecar(): 2018-09-28T15:35:49 - """ + """ xmp_expected_lines = [line.strip() for line in xmp_expected.split("\n")] @@ -464,3 +464,68 @@ def test_xmp_sidecar(): for line_expected, line_got in zip(xmp_expected_lines, xmp_got_lines): assert line_expected == line_got + +def test_xmp_sidecar_keyword_template(): + import osxphotos + + photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB) + photos = photosdb.photos(uuid=[UUID_DICT["xmp"]]) + + xmp_expected = """ + + + + + Girls with pumpkins + Can we carry this? + + + + Kids + Suzy + Katie + + + 2018-09-28T15:35:49.063000-04:00 + + + + + Suzy + Katie + + + + + + + Kids + Test Album + Pumpkin Farm + 2018 + + + + + 2018-09-28T15:35:49 + 2018-09-28T15:35:49 + + + """ + + xmp_expected_lines = [line.strip() for line in xmp_expected.split("\n")] + + xmp_got = photos[0]._xmp_sidecar( + keyword_template=["{folder_album}", "{created.year}"] + ) + xmp_got_lines = [line.strip() for line in xmp_got.split("\n")] + + for line_expected, line_got in zip( + sorted(xmp_expected_lines), sorted(xmp_got_lines) + ): + assert line_expected == line_got diff --git a/tests/test_template.py b/tests/test_template.py index ca4cbcae..bcb2eb74 100644 --- a/tests/test_template.py +++ b/tests/test_template.py @@ -92,18 +92,14 @@ def test_lookup(): """ Test that a lookup is returned for every possible value """ import re import osxphotos - from osxphotos.template import ( - get_template_value, - render_filepath_template, - TEMPLATE_SUBSTITUTIONS, - ) + from osxphotos.template import TEMPLATE_SUBSTITUTIONS photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] for subst in TEMPLATE_SUBSTITUTIONS: lookup_str = re.match(r"\{([^\\,}]+)\}", subst).group(1) - lookup = get_template_value(lookup_str, photo) + lookup = photo.get_template_value(lookup_str) assert lookup or lookup is None @@ -111,14 +107,13 @@ def test_subst(): """ Test that substitutions are correct """ import locale import osxphotos - from osxphotos.template import render_filepath_template locale.setlocale(locale.LC_ALL, "en_US") photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] for template in TEMPLATE_VALUES: - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert rendered[0] == TEMPLATE_VALUES[template] @@ -130,13 +125,12 @@ def test_subst_locale_1(): # osxphotos.template sets local on load so set the environment first # set locale to DE locale.setlocale(locale.LC_ALL, "de_DE.UTF-8") - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] for template in TEMPLATE_VALUES_DEU: - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert rendered[0] == TEMPLATE_VALUES_DEU[template] @@ -155,13 +149,11 @@ def test_subst_locale_2(): os.environ["LC_NUMERIC"] = "de_DE.UTF-8" os.environ["LC_TIME"] = "de_DE.UTF-8" - from osxphotos.template import render_filepath_template - photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] for template in TEMPLATE_VALUES_DEU: - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert rendered[0] == TEMPLATE_VALUES_DEU[template] @@ -169,14 +161,13 @@ def test_subst_default_val(): """ Test substitution with default value specified """ import locale import osxphotos - from osxphotos.template import render_filepath_template locale.setlocale(locale.LC_ALL, "en_US") photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] template = "{place.name.area_of_interest,UNKNOWN}" - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert rendered[0] == "UNKNOWN" @@ -184,14 +175,13 @@ def test_subst_default_val_2(): """ Test substitution with ',' but no default value """ import locale import osxphotos - from osxphotos.template import render_filepath_template locale.setlocale(locale.LC_ALL, "en_US") photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] template = "{place.name.area_of_interest,}" - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert rendered[0] == "_" @@ -199,32 +189,30 @@ def test_subst_unknown_val(): """ Test substitution with unknown value specified """ import locale import osxphotos - from osxphotos.template import render_filepath_template locale.setlocale(locale.LC_ALL, "en_US") photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] template = "{created.year}/{foo}" - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert rendered[0] == "2020/{foo}" assert unknown == ["foo"] template = "{place.name.area_of_interest,}" - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert rendered[0] == "_" def test_subst_double_brace(): """ Test substitution with double brace {{ which should be ignored """ import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] template = "{created.year}/{{foo}}" - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert rendered[0] == "2020/{foo}" assert not unknown @@ -233,14 +221,13 @@ def test_subst_unknown_val_with_default(): """ Test substitution with unknown value specified """ import locale import osxphotos - from osxphotos.template import render_filepath_template locale.setlocale(locale.LC_ALL, "en_US") photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_PLACES) photo = photosdb.photos(uuid=[UUID_DICT["place_dc"]])[0] template = "{created.year}/{foo,bar}" - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert rendered[0] == "2020/{foo,bar}" assert unknown == ["foo"] @@ -249,14 +236,13 @@ def test_subst_multi_1_1_2(): """ Test that substitutions are correct """ # one album, one keyword, two persons import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) photo = photosdb.photos(uuid=[UUID_DICT["1_1_2"]])[0] template = "{created.year}/{album}/{keyword}/{person}" expected = ["2018/Pumpkin Farm/Kids/Katie", "2018/Pumpkin Farm/Kids/Suzy"] - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert sorted(rendered) == sorted(expected) @@ -264,7 +250,6 @@ def test_subst_multi_2_1_1(): """ Test that substitutions are correct """ # 2 albums, 1 keyword, 1 person import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -276,7 +261,7 @@ def test_subst_multi_2_1_1(): "2018/Test Album/Kids/Katie", "2018/Multi Keyword/Kids/Katie", ] - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert sorted(rendered) == sorted(expected) @@ -284,7 +269,6 @@ def test_subst_multi_2_1_1_single(): """ Test that substitutions are correct """ # 2 albums, 1 keyword, 1 person but only do keywords import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -292,7 +276,7 @@ def test_subst_multi_2_1_1_single(): template = "{keyword}" expected = ["Kids"] - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert sorted(rendered) == sorted(expected) @@ -300,7 +284,6 @@ def test_subst_multi_0_2_0(): """ Test that substitutions are correct """ # 0 albums, 2 keywords, 0 persons import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -308,7 +291,7 @@ def test_subst_multi_0_2_0(): template = "{created.year}/{album}/{keyword}/{person}" expected = ["2019/_/wedding/_", "2019/_/flowers/_"] - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert sorted(rendered) == sorted(expected) @@ -316,7 +299,6 @@ def test_subst_multi_0_2_0_single(): """ Test that substitutions are correct """ # 0 albums, 2 keywords, 0 persons, but only do albums import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -324,7 +306,7 @@ def test_subst_multi_0_2_0_single(): template = "{created.year}/{album}" expected = ["2019/_"] - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert sorted(rendered) == sorted(expected) @@ -332,7 +314,6 @@ def test_subst_multi_0_2_0_default_val(): """ Test that substitutions are correct """ # 0 albums, 2 keywords, 0 persons, default vals provided import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -340,7 +321,7 @@ def test_subst_multi_0_2_0_default_val(): template = "{created.year}/{album,NOALBUM}/{keyword,NOKEYWORD}/{person,NOPERSON}" expected = ["2019/NOALBUM/wedding/NOPERSON", "2019/NOALBUM/flowers/NOPERSON"] - rendered, _ = render_filepath_template(template, photo) + rendered, _ = photo.render_template(template) assert sorted(rendered) == sorted(expected) @@ -348,7 +329,6 @@ def test_subst_multi_0_2_0_default_val_unknown_val(): """ Test that substitutions are correct """ # 0 albums, 2 keywords, 0 persons, default vals provided, unknown val in template import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -361,7 +341,7 @@ def test_subst_multi_0_2_0_default_val_unknown_val(): "2019/NOALBUM/wedding/_/{foo}/{baz}", "2019/NOALBUM/flowers/_/{foo}/{baz}", ] - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert sorted(rendered) == sorted(expected) assert unknown == ["foo"] @@ -370,7 +350,6 @@ def test_subst_multi_0_2_0_default_val_unknown_val_2(): """ Test that substitutions are correct """ # 0 albums, 2 keywords, 0 persons, default vals provided, unknown val in template import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_1) # one album, one keyword, two persons @@ -381,7 +360,7 @@ def test_subst_multi_0_2_0_default_val_unknown_val_2(): "2019/NOALBUM/wedding/_/{foo,bar}/{baz,bar}", "2019/NOALBUM/flowers/_/{foo,bar}/{baz,bar}", ] - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert sorted(rendered) == sorted(expected) assert unknown == ["foo"] @@ -389,7 +368,6 @@ def test_subst_multi_0_2_0_default_val_unknown_val_2(): def test_subst_multi_folder_albums_1(): """ Test substitutions for folder_album are correct """ import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_4) @@ -397,7 +375,7 @@ def test_subst_multi_folder_albums_1(): photo = photosdb.photos(uuid=[UUID_DICT["folder_album_1"]])[0] template = "{folder_album}" expected = ["Folder1/SubFolder2/AlbumInFolder"] - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert sorted(rendered) == sorted(expected) assert unknown == [] @@ -405,7 +383,6 @@ def test_subst_multi_folder_albums_1(): def test_subst_multi_folder_albums_2(): """ Test substitutions for folder_album are correct """ import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_4) @@ -413,15 +390,14 @@ def test_subst_multi_folder_albums_2(): photo = photosdb.photos(uuid=[UUID_DICT["folder_album_no_folder"]])[0] template = "{folder_album}" expected = ["Pumpkin Farm", "Test Album"] - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert sorted(rendered) == sorted(expected) assert unknown == [] -def test_subst_multi_folder_albums_3(caplog): +def test_subst_multi_folder_albums_3(): """ Test substitutions for folder_album on < Photos 5 """ import osxphotos - from osxphotos.template import render_filepath_template photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_14_6) @@ -429,6 +405,6 @@ def test_subst_multi_folder_albums_3(caplog): photo = photosdb.photos(uuid=[UUID_DICT["mojave_album_1"]])[0] template = "{folder_album}" expected = ["Folder1/SubFolder2/AlbumInFolder", "Pumpkin Farm", "Test Album (1)"] - rendered, unknown = render_filepath_template(template, photo) + rendered, unknown = photo.render_template(template) assert sorted(rendered) == sorted(expected) assert unknown == []