Refactored phototemplate.py to add PATH_SEP option

This commit is contained in:
Rhet Turnbull 2020-11-08 16:09:51 -08:00
parent a6231e29ff
commit 3636fcbc76
5 changed files with 264 additions and 64 deletions

138
README.md
View File

@ -388,30 +388,72 @@ option to re-export the entire library thus rebuilding the
** Templating System **
Several options, such as --directory, allow you to specify a template which
will be rendered to substitute template fields with values from the photo. For
example, '{created.month}' would be replaced with the month name of the photo
creation date. e.g. 'November'. The general format for a template is
'{TEMPLATE_FIELD[,[DEFAULT]]}'. The ',' and DEFAULT value are optional. If
TEMPLATE_FIELD results in a null (empty) value, the default is '_'. You may
specify an alternate default value by appending ',DEFAULT' after
template_field. e.g. '{title,no_title}' would result in 'no_title' if the
photo had no title. You may include other text in the template string outside
the {} and use more than one template field, e.g. '{created.year} -
{created.month}' (e.g. '2020 - November'). Some template fields such as 'hdr'
are boolean and resolve to True or False. These take the form:
'{TEMPLATE_FIELD?VALUE_IF_TRUE,VALUE_IF_FALSE}', e.g. '{hdr?is_hdr,not_hdr}'.
Several options, such as --directory, allow you to specify a template which
will be rendered to substitute template fields with values from the photo.
For example, '{created.month}' would be replaced with the month name of the
photo creation date. e.g. 'November'.
The general format for a template is '{TEMPLATE_FIELD[,[DEFAULT]]}'. Some
templates have optional modifiers in form
'{[[DELIM]+]TEMPLATE_FIELD[(PATH_SEP)][?VALUE_IF_TRUE][,[DEFAULT]]}'
The ',' and DEFAULT value are optional. If TEMPLATE_FIELD results in a null
(empty) value, the default is '_'. You may specify an alternate default
value by appending ',DEFAULT' after template_field. e.g. '{title,no_title}'
would result in 'no_title' if the photo had no title. You may include other
text in the template string outside the {} and use more than one template
field, e.g. '{created.year} - {created.month}' (e.g. '2020 - November').
Some template fields such as 'hdr' are boolean and resolve to True or False.
These take the form: '{TEMPLATE_FIELD?VALUE_IF_TRUE,VALUE_IF_FALSE}', e.g.
{hdr?is_hdr,not_hdr} which would result in 'is_hdr' if photo is an HDR image
and 'not_hdr' otherwise.
Some template fields such as 'folder_template' are "path-like" in that they
join multiple elements into a single path-like string. For example, if photo
is in album Album1 in folder Folder1, '{folder_album}` results in
'Folder1/Album1'. This is so these template fields may be used as paths in
--directory. If you intend to use such a field as a string, e.g. in the
filename, you may specify a different path separator using the form:
'{TEMPLATE_FIELD(PATH_SEP)}'. For example, using the example above,
'{folder_album(-)}' would result in 'Folder1-Album1' and '{folder_album()}'
would result in 'Folder1Album1'.
Some templates may resolve to more than one value. For example, a photo can
have multiple keywords so '{keyword}' can result in multiple values. If used
in a filename or directory, these templates may result in more than one copy
of the photo being exported. For example, if photo has keywords "foo" and
"bar", --directory '{keyword}' will result in copies of the photo being
exported to 'foo/image_name.jpeg' and 'bar/image_name.jpeg'.
Multi-value template fields such as '{keyword}' may be expanded 'in place'
with an optional delimiter using the template form '{DELIM+TEMPLATE_FIELD}'.
For example, a photo with keywords 'foo' and 'bar':
'{keyword}' renders to 'foo' and 'bar'
'{,+keyword}' renders to: 'foo,bar'
'{; +keyword}' renders to: 'foo; bar'
'{+keyword}' renders to 'foobar'
Some template fields such as '{media_type}' use the 'DEFAULT' value to allow
customization of the output. For example, '{media_type}' resolves to the
special media type of the photo such as 'panorama' or 'selfie'. You may use
the 'DEFAULT' value to override these in form:
'{media_type,video=vidéo;time_lapse=vidéo_accélérée}'. In this example, if
photo is a time_lapse photo, 'media_type' would resolve to 'vidéo_accélérée'
instead of 'time_lapse' and video would resolve to 'vidéo' if photo is an
ordinary video.
With the --directory and --filename options you may specify a template for the
export directory or filename, respectively. The directory will be appended to
the export path specified in the export DEST argument to export. For example,
if template is '{created.year}/{created.month}', and export desitnation DEST
if template is '{created.year}/{created.month}', and export destination DEST
is '/Users/maria/Pictures/export', the actual export directory for a photo
would be '/Users/maria/Pictures/export/2020/March' if the photo was created in
March 2020. Some template substitutions may result in more than one value, for
example '{album}' if photo is in more than one album or '{keyword}' if photo
has more than one keyword. In this case, more than one copy of the photo will
be exported, each in a separate directory or with a different filename.
March 2020.
The templating system may also be used with the --keyword-template option to
set keywords on export (with --exiftool or --sidecar), for example, to set a
@ -1424,11 +1466,13 @@ If overwrite=False and increment=False, export will fail if destination file alr
#### <a name="rendertemplate">`render_template()`</a>
`render_template(template_str, none_str = "_", path_sep = None, expand_inplace = False, inplace_sep = None, filename=False, dirname=False, replacement=":",)`
Render template string for photo. none_str is used if template substitution results in None value and no default specified.
- `template_str`: str in form "{name,DEFAULT}" where name is one of the values in table below. The "," and default value that follows are optional. If specified, "DEFAULT" will be used if "name" is None. This is useful for values which are not always present, for example reverse geolocation data.
`render_template(template_str, none_str = "_", path_sep = None, expand_inplace = False, inplace_sep = None, filename=False, dirname=False, replacement=":",)`
Render template string for photo. none_str is used if template substitution results in None value and no default specified.
- `template_str`: str in format "{[[DELIM]+]name[(PATH_SEP)][?TRUE_VALUE][,[DEFAULT]]}" where name is one of the values in the [Template Substitutions](#template-substitutions) table. See notes below regarding specific details of the syntax.
- `none_str`: optional str to use as substitution when template value is None and no default specified in the template string. default is "_".
- `path_sep`: optional character to use as path separator, default is os.path.sep
- `path_sep`: optional character to use as path separator, default is `os.path.sep`
- `expand_inplace`: expand multi-valued substitutions in-place as a single string instead of returning individual strings
- `inplace_sep`: optional string to use as separator between multi-valued keywords with expand_inplace; default is ','
- `filename`: if True, template output will be sanitized to produce valid file name
@ -1445,6 +1489,56 @@ e.g. `render_template("{created.year}/{{foo}}", photo)` would return `(["2020/{f
Some substitutions, notably `album`, `keyword`, and `person` could return multiple values, hence a new string will be return for each possible substitution (hence why a list of rendered strings is returned). For example, a photo in 2 albums: 'Vacation' and 'Family' would result in the following rendered values if template was "{created.year}/{album}" and created.year == 2020: `["2020/Vacation","2020/Family"]`
The template field format contains optional modifiers:
`"{[[DELIM]+]name[(PATH_SEP)][?TRUE_VALUE][,[DEFAULT]]}"`
`DELIM`: optional delimiter string to use when expanding multi-valued template values in-place
`+`: If present before template `name`, expands the template in place. If `DELIM` not provided, values are joined with no delimiter.
e.g. if Photo keywords are `["foo","bar"]`:
- `"{keyword}"` renders to `["foo", "bar"]`
- `"{,+keyword}"` renders to: `["foo,bar"]`
- `"{; +keyword}"` renders to: `["foo; bar"]`
- `"{+keyword}"` renders to `["foobar"]`
`PATH_SEP`: optional path separator to use when joining path like fields, for example `{folder_album}`. May also be provided as `path_sep` argument in `render_template()`. If provided both in the call to `render_template()` and in the template itself, the value in the template string takes precedence. If not provided in either the template string or in `path_sep` argument, defaults to `os.path.sep`.
e.g. If Photo is in `Album1` in `Folder1`:
- `"{folder_album}"` renders to `["Folder1/Album1"]`
- `"{folder_album(:)}"` renders to `["Folder1:Album1"]`
- `"{folder_album()}"` renders to `["Folder1Album1"]`
`?TRUE_VALUE`: optional value to use if name is boolean-type field which evaluates to true. For example `"{hdr}"` evaluates to True if photo is an high dynamic range (HDR) image and False otherwise. In these types of fields, use `?TRUE_VALUE` to provide the value if True and `,DEFAULT` to provide the value of False.
e.g. if photo is an HDR image,
- `"{hdr?ISHDR,NOTHDR}"` renders to `["ISHDR"]`
and if it is not an HDR image,
- `"{hdr?ISHDR,NOTHDR}"` renders to `["NOTHDR"]`
Either or both `TRUE_VALUE` or `DEFAULT` (False value) may be empty which would result in empty string `[""]` when rendered.
`,DEFAULT`: optional default value to use if the template name has no value. This modifier is also used for the value if False for boolean-type fields (see above) as well as to hold a sub-template for values like `{created.strftime}`. If no default value provided, "_" is used. May also be provided in the `none_str` argument to `render_template()`. If provided both in the template string and in `none_str`, the value in the template string takes precedence.
e.g., if photo has no title set,
- `"{title}"` renders to ["_"]
- `"{title,I have no title}"` renders to `["I have no title"]`
Template fields such as `created.strftime` use the DEFAULT value to pass the template to use for `strftime`.
e.g., if photo date is 4 February 2020, 19:07:38,
- `"{created.strftime,%Y-%m-%d-%H%M%S}"` renders to `["2020-02-04-190738"]`
Some template fields such as `"{media_type}"` use the `DEFAULT` value to allow customization of the output. For example, `"{media_type}"` resolves to the special media type of the photo such as `panorama` or `selfie`. You may use the `DEFAULT` value to override these in form: `"{media_type,video=vidéo;time_lapse=vidéo_accélérée}"`. In this example, if photo was a time_lapse photo, `media_type` would resolve to `vidéo_accélérée` instead of `time_lapse`.
See [Template Substitutions](#template-substitutions) for additional details.
### ExifInfo

View File

@ -159,20 +159,64 @@ class ExportCommand(click.Command):
formatter.write_text("** Templating System **")
formatter.write("\n")
formatter.write_text(
"Several options, such as --directory, allow you to specify a template "
+ "which will be rendered to substitute template fields with values from the photo. "
+ "For example, '{created.month}' would be replaced with the month name of the photo creation date. "
+ "e.g. 'November'. "
+ "The general format for a template is '{TEMPLATE_FIELD[,[DEFAULT]]}'. "
+ "The ',' and DEFAULT value are optional. "
+ "If TEMPLATE_FIELD results in a null (empty) value, the default is '_'. "
+ "You may specify an alternate default value by appending ',DEFAULT' after template_field. "
+ "e.g. '{title,no_title}' would result in 'no_title' if the photo had no title. "
+ "You may include other text in the template string outside the {} and use more than "
+ "one template field, e.g. '{created.year} - {created.month}' (e.g. '2020 - November'). "
+ "Some template fields such as 'hdr' are boolean and resolve to True or False. "
+ "These take the form: '{TEMPLATE_FIELD?VALUE_IF_TRUE,VALUE_IF_FALSE}', e.g. "
+ "'{hdr?is_hdr,not_hdr}'."
"""
Several options, such as --directory, allow you to specify a template
which will be rendered to substitute template fields with values from the photo.
For example, '{created.month}' would be replaced with the month name of the photo creation date.
e.g. 'November'.
\n
The general format for a template is '{TEMPLATE_FIELD[,[DEFAULT]]}'.
Some templates have optional modifiers in form
'{[[DELIM]+]TEMPLATE_FIELD[(PATH_SEP)][?VALUE_IF_TRUE][,[DEFAULT]]}'
\n
The ',' and DEFAULT value are optional.
If TEMPLATE_FIELD results in a null (empty) value, the default is '_'.
You may specify an alternate default value by appending ',DEFAULT' after template_field.
e.g. '{title,no_title}' would result in 'no_title' if the photo had no title.
You may include other text in the template string outside the {} and use more than
one template field, e.g. '{created.year} - {created.month}' (e.g. '2020 - November').
\n
Some template fields such as 'hdr' are boolean and resolve to True or False.
These take the form: '{TEMPLATE_FIELD?VALUE_IF_TRUE,VALUE_IF_FALSE}', e.g.
{hdr?is_hdr,not_hdr} which would result in 'is_hdr' if photo is an HDR
image and 'not_hdr' otherwise.
\n
Some template fields such as 'folder_template' are "path-like" in that they join
multiple elements into a single path-like string. For example, if photo is in
album Album1 in folder Folder1, '{folder_album}` results in 'Folder1/Album1'.
This is so these template fields may be used as paths in --directory.
If you intend to use such a field as a string, e.g. in the filename, you may specify
a different path separator using the form: '{TEMPLATE_FIELD(PATH_SEP)}'.
For example, using the example above, '{folder_album(-)}' would result in
'Folder1-Album1' and '{folder_album()}' would result in
'Folder1Album1'.
\n
Some templates may resolve to more than one value. For example, a photo can have
multiple keywords so '{keyword}' can result in multiple values. If used in a filename
or directory, these templates may result in more than one copy of the photo being exported.
For example, if photo has keywords "foo" and "bar", --directory '{keyword}' will result in
copies of the photo being exported to 'foo/image_name.jpeg' and 'bar/image_name.jpeg'.
\n
Multi-value template fields such as '{keyword}' may be expanded 'in place' with an optional
delimiter using the template form '{DELIM+TEMPLATE_FIELD}'. For example, a photo with
keywords 'foo' and 'bar':
\n
'{keyword}' renders to 'foo' and 'bar'
\n
'{,+keyword}' renders to: 'foo,bar'
\n
'{; +keyword}' renders to: 'foo; bar'
\n
'{+keyword}' renders to 'foobar'
\n
Some template fields such as '{media_type}' use the 'DEFAULT' value to allow customization
of the output. For example, '{media_type}' resolves to the special media type of the
photo such as 'panorama' or 'selfie'. You may use the 'DEFAULT' value to override
these in form: '{media_type,video=vidéo;time_lapse=vidéo_accélérée}'.
In this example, if photo is a time_lapse photo, 'media_type' would resolve to
'vidéo_accélérée' instead of 'time_lapse' and video would resolve to 'vidéo' if photo
is an ordinary video.
"""
)
formatter.write("\n")
formatter.write_text(
@ -180,14 +224,10 @@ class ExportCommand(click.Command):
+ "export directory or filename, respectively. "
+ "The directory will be appended to the export path specified "
+ "in the export DEST argument to export. For example, if template is "
+ "'{created.year}/{created.month}', and export desitnation DEST is "
+ "'{created.year}/{created.month}', and export destination DEST is "
+ "'/Users/maria/Pictures/export', "
+ "the actual export directory for a photo would be '/Users/maria/Pictures/export/2020/March' "
+ "if the photo was created in March 2020. "
+ "Some template substitutions may result in more than one value, for example '{album}' if "
+ "photo is in more than one album or '{keyword}' if photo has more than one keyword. "
+ "In this case, more than one copy of the photo will be exported, each in a separate directory "
+ "or with a different filename."
)
formatter.write("\n")
formatter.write_text(

View File

@ -1,4 +1,4 @@
""" version info """
__version__ = "0.36.12"
__version__ = "0.36.13"

View File

@ -166,7 +166,7 @@ class PhotoTemplate:
Args:
template: str template
none_str: str to use default for None values, default is '_'
path_sep: optional character to use as path separator, default is os.path.sep
path_sep: optional string to use as path separator, default is os.path.sep
expand_inplace: expand multi-valued substitutions in-place as a single string
instead of returning individual strings
inplace_sep: optional string to use as separator between multi-valued keywords
@ -181,8 +181,6 @@ class PhotoTemplate:
if path_sep is None:
path_sep = os.path.sep
elif path_sep is not None and len(path_sep) != 1:
raise ValueError(f"path_sep must be single character: {path_sep}")
if inplace_sep is None:
inplace_sep = ","
@ -196,9 +194,17 @@ class PhotoTemplate:
# there would be 6 possible renderings (2 albums x 3 persons)
# regex to find {template_field,optional_default} in strings
# for explanation of regex see https://regex101.com/r/YFpWsn/1
# pylint: disable=anomalous-backslash-in-string
regex = r"(?<!\{)\{([^}]*\+)?([^\\,}+\?]+)(\?[^\\,}]*)?(,[\w\=\;\-\%. ]*)?(?=\}(?!\}))\}"
regex = (
r"(?<!\{)\{" # match { but not {{
+ r"([^}]*\+)?" # group 1: optional DELIM+
+ r"([^\\,}+\?]+)" # group 2: field name
+ r"(\([^{}\)]*\))?" # group 3: optional (PATH_SEP)
+ r"(\?[^\\,}]*)?" # group 4: optional ?TRUE_VALUE for boolean fields
+ r"(,[\w\=\;\-\%. ]*)?" # group 5: optional ,DEFAULT
+ r"(?=\}(?!\}))\}" # match } but not }}
)
if type(template) is not str:
raise TypeError(f"template must be type str, not {type(template)}")
@ -219,21 +225,24 @@ class PhotoTemplate:
# closure to capture photo, none_str, filename, dirname in subst
def subst(matchobj):
groups = len(matchobj.groups())
if groups == 4:
if groups == 5:
delim = matchobj.group(1)
field = matchobj.group(2)
bool_val = matchobj.group(3)
default = matchobj.group(4)
path_sep = matchobj.group(3)
bool_val = matchobj.group(4)
default = matchobj.group(5)
# drop the comma on default
default_val = default[1:] if default is not None else None
# drop the '+' on delim
delim = delim[:-1] if delim is not None else None
# drop () from path_sep
path_sep = path_sep.strip("()") if path_sep is not None else None
# drop the ? on bool_val
bool_val = bool_val[1:] if bool_val is not None else None
# drop the comma on default
default_val = default[1:] if default is not None else None
try:
val = get_func(field, default_val, bool_val, delim)
val = get_func(field, default_val, bool_val, delim, path_sep)
except ValueError:
return matchobj.group(0)
@ -284,12 +293,15 @@ class PhotoTemplate:
for field in MULTI_VALUE_SUBSTITUTIONS:
# Build a regex that matches only the field being processed
re_str = (
r"(?<!\{)\{"
+ r"([^}]*\+)?" # group 1, optional delim/expand in place
r"(?<!\{)\{" # match { but not {{
+ r"([^}]*\+)?" # group 1: optional DELIM+
+ r"("
+ field # group 2 (field name)
+ field # group 2: field name
+ r")"
+ r"(\?[^\\,}]*)?(,[\w\=\;\-\%. ]*)?(?=\}(?!\}))\}"
+ r"(\([^{}\)]*\))?" # group 3: optional (PATH_SEP)
+ r"(\?[^\\,}]*)?" # group 4: optional ?TRUE_VALUE for boolean fields
+ r"(,[\w\=\;\-\%. ]*)?" # group 5: optional ,DEFAULT
+ r"(?=\}(?!\}))\}" # match } but not }}
)
regex_multi = re.compile(re_str)
@ -299,6 +311,11 @@ class PhotoTemplate:
for str_template in rendered_strings:
matches = regex_multi.search(str_template)
if matches:
path_sep = (
matches.group(3).strip("()")
if matches.group(3) is not None
else path_sep
)
values = self.get_template_value_multi(
field,
path_sep,
@ -308,14 +325,10 @@ class PhotoTemplate:
)
if expand_inplace or matches.group(1) is not None:
delim = (
matches.group(1)[:-1]
if matches.group(1) is not None
else inplace_sep
matches.group(1)[:-1] if matches.group(1) is not None else inplace_sep
)
# instead of returning multiple strings, join values into a single string
val = (
delim.join(sorted(values)) if values and values[0] else None
)
val = delim.join(sorted(values)) if values and values[0] else None
def lookup_template_value_multi(lookup_value, *_):
""" Closure passed to make_subst_function get_func
@ -389,6 +402,7 @@ class PhotoTemplate:
default,
bool_val=None,
delim=None,
path_sep=None,
filename=False,
dirname=False,
replacement=":",
@ -398,6 +412,9 @@ class PhotoTemplate:
Args:
field: template field to find value for.
default: the default value provided by the user
bool_val: True value if expression is boolean
delim: delimiter for expand in place
path_sep: path separator for fields that are path-like
filename: if True, template output will be sanitized to produce valid file name
dirname: if True, template output will be sanitized to produce valid directory name
replacement: str, value to replace any illegal file path characters with; default = ":"

View File

@ -496,6 +496,25 @@ def test_subst_multi_folder_albums_1():
assert unknown == []
def test_subst_multi_folder_albums_1_path_sep():
""" Test substitutions for folder_album are correct with custom PATH_SEP """
import osxphotos
photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_7)
# photo in an album in a folder
photo = photosdb.photos(uuid=[UUID_DICT["folder_album_1"]])[0]
template = "{folder_album(:)}"
expected = [
"2018-10 - Sponsion, Museum, Frühstück, Römermuseum",
"2019-10/11 Paris Clermont",
"Folder1:SubFolder2:AlbumInFolder",
]
rendered, unknown = photo.render_template(template)
assert sorted(rendered) == sorted(expected)
assert unknown == []
def test_subst_multi_folder_albums_2():
""" Test substitutions for folder_album are correct """
import osxphotos
@ -511,6 +530,21 @@ def test_subst_multi_folder_albums_2():
assert unknown == []
def test_subst_multi_folder_albums_2_path_sep():
""" Test substitutions for folder_album are correct with custom PATH_SEP """
import osxphotos
photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_15_7)
# photo in an album in a folder
photo = photosdb.photos(uuid=[UUID_DICT["folder_album_no_folder"]])[0]
template = "{folder_album(:)}"
expected = ["Pumpkin Farm", "Test Album"]
rendered, unknown = photo.render_template(template)
assert sorted(rendered) == sorted(expected)
assert unknown == []
def test_subst_multi_folder_albums_3():
""" Test substitutions for folder_album on < Photos 5 """
import osxphotos
@ -526,6 +560,21 @@ def test_subst_multi_folder_albums_3():
assert unknown == []
def test_subst_multi_folder_albums_3_path_sep():
""" Test substitutions for folder_album on < Photos 5 with custom PATH_SEP """
import osxphotos
photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB_14_6)
# photo in an album in a folder
photo = photosdb.photos(uuid=[UUID_DICT["mojave_album_1"]])[0]
template = "{folder_album(:)}"
expected = ["Folder1:SubFolder2:AlbumInFolder", "Pumpkin Farm", "Test Album (1)"]
rendered, unknown = photo.render_template(template)
assert sorted(rendered) == sorted(expected)
assert unknown == []
def test_subst_strftime():
""" Test that strftime substitutions are correct """
import locale
@ -687,4 +736,4 @@ def test_expand_in_place_with_delim_single_value():
for template in TEMPLATE_VALUES_TITLE:
rendered, _ = photo.render_template(template)
assert sorted(rendered) == sorted(TEMPLATE_VALUES_TITLE[template])
assert sorted(rendered) == sorted(TEMPLATE_VALUES_TITLE[template])