Added --xattr-template, closes #242

This commit is contained in:
Rhet Turnbull
2020-12-31 12:33:15 -08:00
parent 34bb7f2cdc
commit bf2a55d7f6
7 changed files with 297 additions and 41 deletions

View File

@@ -412,20 +412,30 @@ Options:
could specify --description-template could specify --description-template
"{descr} exported with osxphotos on "{descr} exported with osxphotos on
{today.date}" See Templating System below. {today.date}" See Templating System below.
--finder-tag-template TEMPLATE Set Finder tags to TEMPLATE. These tags can --finder-tag-template TEMPLATE Set MacOS Finder tags to TEMPLATE. These
be searched in the Finder or Spotlight with tags can be searched in the Finder or
'tag:tagname' format. For example, '-- Spotlight with 'tag:tagname' format. For
finder-tag-template "{label}"' to set Finder example, '--finder-tag-template "{label}"'
tags to photo labels. You may specify to set Finder tags to photo labels. You may
multiple TEMPLATE values by using '--finder- specify multiple TEMPLATE values by using '
tag-template' multiple times. See also '-- --finder-tag-template' multiple times. See
finder-tag-keywords and Extended Attributes also '--finder-tag-keywords and Extended
below.'. Attributes below.'.
--finder-tag-keywords Set Finder tags to keywords; any keywords --finder-tag-keywords Set MacOS Finder tags to keywords; any
specified via '--keyword-template', '-- keywords specified via '--keyword-template',
person-keyword', etc. will also be used as '--person-keyword', etc. will also be used
Finder tags. See also '--finder-tag-template as Finder tags. See also '--finder-tag-
and Extended Attributes below.'. template and Extended Attributes below.'.
--xattr-template ATTRIBUTE TEMPLATE
Set extended attribute ATTRIBUTE to TEMPLATE
value. Valid attributes are: 'authors',
'comment', 'copyright', 'description',
'findercomment', 'headline', 'keywords'. For
example, to set Finder comment to the
photo's title and description: '--xattr-
template findercomment "{title}; {descr}"
See Extended Attributes below for additional
details on this option.
--directory DIRECTORY Optional template for specifying name of --directory DIRECTORY Optional template for specifying name of
output directory in the form output directory in the form
'{name,DEFAULT}'. See below for additional '{name,DEFAULT}'. See below for additional
@@ -530,21 +540,50 @@ option to re-export the entire library thus rebuilding the
** Extended Attributes ** ** Extended Attributes **
Some options (currently '--finder-tag-template' and '--finder-tag-keywords') Some options (currently '--finder-tag-template', '--finder-tag-keywords',
write additional metadata to extended attributes in the file. These options '-xattr-template') write additional metadata to extended attributes in the
will only work if the destination filesystem supports extended attributes file. These options will only work if the destination filesystem supports
(most do). For example, --finder-tag-keyword writes all keywords (including extended attributes (most do). For example, --finder-tag-keyword writes all
any specified by '--keyword-template' or other options) to Finder tags that keywords (including any specified by '--keyword-template' or other options) to
are searchable in Spotlight using the syntax: 'tag:tagname'. For example, if Finder tags that are searchable in Spotlight using the syntax: 'tag:tagname'.
you have images with keyword "Travel" then using '--finder-tag-keywords' you For example, if you have images with keyword "Travel" then using '--finder-
could quickly find those images in the Finder by typing 'tag:Travel' in the tag-keywords' you could quickly find those images in the Finder by typing
Spotlight search bar. Finder tags are written to the 'tag:Travel' in the Spotlight search bar. Finder tags are written to the
'com.apple.metadata:_kMDItemUserTags' extended attribute. Unlike EXIF 'com.apple.metadata:_kMDItemUserTags' extended attribute. Unlike EXIF
metadata, extended attributes do not modify the actual file. Most cloud metadata, extended attributes do not modify the actual file. Most cloud
storage services do not synch extended attributes. Dropbox does sync them and storage services do not synch extended attributes. Dropbox does sync them and
any changes to a file's extended attributes will cause Dropbox to re-sync the any changes to a file's extended attributes will cause Dropbox to re-sync the
files. files.
The following attributes may be used with '--xattr-template':
authors The author, or authors, of the contents of the file. A list
of strings. (com.apple.metadata:kMDItemAuthors)
comment A comment related to the file. This differs from the Finder
comment, kMDItemFinderComment. A string.
(com.apple.metadata:kMDItemComment)
copyright The copyright owner of the file contents. A string.
(com.apple.metadata:kMDItemCopyright)
description A description of the content of the resource. The
description may include an abstract, table of contents,
reference to a graphical representation of content or a free-
text account of the content. A string.
(com.apple.metadata:kMDItemDescription)
findercomment Finder comments for this file. A string.
(com.apple.metadata:kMDItemFinderComment)
headline A publishable entry providing a synopsis of the contents of
the file. A string. (com.apple.metadata:kMDItemHeadline)
keywords Keywords associated with this file. For example, “Birthday”,
“Important”, etc. This differs from Finder tags
(_kMDItemUserTags) which are keywords/tags shown in the
Finder and searchable in Spotlight using "tag:tag_name". A
list of strings. (com.apple.metadata:kMDItemKeywords)
For additional information on extended attributes see: https://developer.apple
.com/documentation/coreservices/file_metadata/mditem/common_metadata_attribute
_keys
** Templating System ** ** Templating System **

View File

@@ -11,21 +11,23 @@ import time
import unicodedata import unicodedata
import click import click
import osxmetadata
import yaml import yaml
import osxmetadata
import osxphotos import osxphotos
from ._constants import ( from ._constants import (
_EXIF_TOOL_URL, _EXIF_TOOL_URL,
_OSXPHOTOS_NONE_SENTINEL,
_PHOTOS_4_VERSION, _PHOTOS_4_VERSION,
_UNKNOWN_PLACE, _UNKNOWN_PLACE,
_OSXPHOTOS_NONE_SENTINEL,
CLI_COLOR_ERROR, CLI_COLOR_ERROR,
CLI_COLOR_WARNING, CLI_COLOR_WARNING,
DEFAULT_EDITED_SUFFIX, DEFAULT_EDITED_SUFFIX,
DEFAULT_JPEG_QUALITY, DEFAULT_JPEG_QUALITY,
DEFAULT_ORIGINAL_SUFFIX, DEFAULT_ORIGINAL_SUFFIX,
EXTENDED_ATTRIBUTE_NAMES,
EXTENDED_ATTRIBUTE_NAMES_QUOTED,
SIDECAR_EXIFTOOL, SIDECAR_EXIFTOOL,
SIDECAR_JSON, SIDECAR_JSON,
SIDECAR_XMP, SIDECAR_XMP,
@@ -186,7 +188,7 @@ class ExportCommand(click.Command):
formatter.write("\n") formatter.write("\n")
formatter.write_text( formatter.write_text(
""" """
Some options (currently '--finder-tag-template' and '--finder-tag-keywords') write Some options (currently '--finder-tag-template', '--finder-tag-keywords', '-xattr-template') write
additional metadata to extended attributes in the file. These options will only work additional metadata to extended attributes in the file. These options will only work
if the destination filesystem supports extended attributes (most do). if the destination filesystem supports extended attributes (most do).
For example, --finder-tag-keyword writes all keywords (including any specified by '--keyword-template' For example, --finder-tag-keyword writes all keywords (including any specified by '--keyword-template'
@@ -197,9 +199,24 @@ Finder tags are written to the 'com.apple.metadata:_kMDItemUserTags' extended at
Unlike EXIF metadata, extended attributes do not modify the actual file. Most cloud storage services Unlike EXIF metadata, extended attributes do not modify the actual file. Most cloud storage services
do not synch extended attributes. Dropbox does sync them and any changes to a file's extended attributes do not synch extended attributes. Dropbox does sync them and any changes to a file's extended attributes
will cause Dropbox to re-sync the files. will cause Dropbox to re-sync the files.
The following attributes may be used with '--xattr-template':
""" """
) )
formatter.write_dl(
[
(
attr,
f"{osxmetadata.ATTRIBUTES[attr].help} ({osxmetadata.ATTRIBUTES[attr].constant})",
)
for attr in EXTENDED_ATTRIBUTE_NAMES
]
)
formatter.write("\n")
formatter.write_text(
"For additional information on extended attributes see: https://developer.apple.com/documentation/coreservices/file_metadata/mditem/common_metadata_attribute_keys"
)
formatter.write("\n\n") formatter.write("\n\n")
formatter.write_text("** Templating System **") formatter.write_text("** Templating System **")
formatter.write("\n") formatter.write("\n")
@@ -1490,6 +1507,17 @@ def query(
help="Set MacOS Finder tags to keywords; any keywords specified via '--keyword-template', '--person-keyword', etc. " help="Set MacOS Finder tags to keywords; any keywords specified via '--keyword-template', '--person-keyword', etc. "
"will also be used as Finder tags. See also '--finder-tag-template and Extended Attributes below.'.", "will also be used as Finder tags. See also '--finder-tag-template and Extended Attributes below.'.",
) )
@click.option(
"--xattr-template",
nargs=2,
metavar="ATTRIBUTE TEMPLATE",
multiple=True,
help="Set extended attribute ATTRIBUTE to TEMPLATE value. Valid attributes are: "
f"{', '.join(EXTENDED_ATTRIBUTE_NAMES_QUOTED)}. "
"For example, to set Finder comment to the photo's title and description: "
"'--xattr-template findercomment \"{title}; {descr}\" "
"See Extended Attributes below for additional details on this option.",
)
@click.option( @click.option(
"--directory", "--directory",
metavar="DIRECTORY", metavar="DIRECTORY",
@@ -1632,6 +1660,7 @@ def export(
description_template, description_template,
finder_tag_template, finder_tag_template,
finder_tag_keywords, finder_tag_keywords,
xattr_template,
current_name, current_name,
convert_to_jpeg, convert_to_jpeg,
jpeg_quality, jpeg_quality,
@@ -1770,6 +1799,7 @@ def export(
description_template = cfg.description_template description_template = cfg.description_template
finder_tag_template = cfg.finder_tag_template finder_tag_template = cfg.finder_tag_template
finder_tag_keywords = cfg.finder_tag_keywords finder_tag_keywords = cfg.finder_tag_keywords
xattr_template = cfg.xattr_template
current_name = cfg.current_name current_name = cfg.current_name
convert_to_jpeg = cfg.convert_to_jpeg convert_to_jpeg = cfg.convert_to_jpeg
jpeg_quality = cfg.jpeg_quality jpeg_quality = cfg.jpeg_quality
@@ -1884,6 +1914,19 @@ def export(
) )
raise click.Abort() raise click.Abort()
if xattr_template:
for attr, _ in xattr_template:
if attr not in EXTENDED_ATTRIBUTE_NAMES:
click.echo(
click.style(
f"Invalid attribute '{attr}' for --xattr-template; "
f"valid values are {', '.join(EXTENDED_ATTRIBUTE_NAMES_QUOTED)}",
fg=CLI_COLOR_ERROR,
),
err=True,
)
raise click.Abort()
if save_config: if save_config:
verbose_(f"Saving options to file {save_config}") verbose_(f"Saving options to file {save_config}")
cfg.write_to_file(save_config) cfg.write_to_file(save_config)
@@ -2160,18 +2203,21 @@ def export(
) )
results += export_results results += export_results
# all photo files (not including sidecars) that are part of this export set
# used below for applying Finder tags, etc.
photo_files = set(
export_results.exported
+ export_results.new
+ export_results.updated
+ export_results.exif_updated
+ export_results.converted_to_jpeg
+ export_results.skipped
)
if finder_tag_keywords or finder_tag_template: if finder_tag_keywords or finder_tag_template:
files = set(
export_results.exported
+ export_results.new
+ export_results.updated
+ export_results.exif_updated
+ export_results.converted_to_jpeg
+ export_results.skipped
)
tags_written, tags_skipped = write_finder_tags( tags_written, tags_skipped = write_finder_tags(
p, p,
files, photo_files,
keywords=finder_tag_keywords, keywords=finder_tag_keywords,
keyword_template=keyword_template, keyword_template=keyword_template,
album_keyword=album_keyword, album_keyword=album_keyword,
@@ -2182,6 +2228,13 @@ def export(
results.xattr_written.extend(tags_written) results.xattr_written.extend(tags_written)
results.xattr_skipped.extend(tags_skipped) results.xattr_skipped.extend(tags_skipped)
if xattr_template:
xattr_written, xattr_skipped = write_extended_attributes(
p, photo_files, xattr_template
)
results.xattr_written.extend(xattr_written)
results.xattr_skipped.extend(xattr_skipped)
if fp is not None: if fp is not None:
fp.close() fp.close()
@@ -3444,5 +3497,64 @@ def write_finder_tags(
return (written, skipped) return (written, skipped)
def write_extended_attributes(photo, files, xattr_template):
""" Writes extended attributes to exported files
Args:
photo: a PhotoInfo object
xattr_template: list of tuples: (attribute name, attribute template)
Returns:
tuple(list of file paths that were updated with new attributes, list of file paths skipped because attributes didn't need updating)
"""
attributes = {}
for xattr, template_str in xattr_template:
rendered, unmatched = photo.render_template(
template_str, none_str=_OSXPHOTOS_NONE_SENTINEL, path_sep="/"
)
if unmatched:
click.echo(
click.style(
f"Warning: unmatched template substitution for template: {template_str} {unmatched}",
fg=CLI_COLOR_WARNING,
),
err=True,
)
# filter out any template values that didn't match by looking for sentinel
rendered = [
value for value in rendered if _OSXPHOTOS_NONE_SENTINEL not in value
]
try:
attributes[xattr].extend(rendered)
except KeyError:
attributes[xattr] = rendered
written = set()
skipped = set()
for f in files:
md = osxmetadata.OSXMetaData(f)
for attr, value in attributes.items():
islist = osxmetadata.ATTRIBUTES[attr].list
if value:
value = ", ".join(value) if not islist else sorted(value)
file_value = md.get_attribute(attr)
if file_value and islist:
file_value = sorted(file_value)
if (not file_value and not value) or file_value == value:
# if both not set or both equal, nothing to do
# get_attribute returns None if not set and value will be [] if not set so can't directly compare
verbose_(f"Skipping extended attribute {attr} for {f}: nothing to do")
skipped.add(f)
else:
verbose_(f"Writing extended attribute {attr} to {f}")
md.set_attribute(attr, value)
written.add(f)
return list(written), [f for f in skipped if f not in written]
if __name__ == "__main__": if __name__ == "__main__":
cli() # pylint: disable=no-value-for-parameter cli() # pylint: disable=no-value-for-parameter

View File

@@ -183,3 +183,15 @@ CLI_COLOR_WARNING = "yellow"
SIDECAR_JSON = 0x1 SIDECAR_JSON = 0x1
SIDECAR_EXIFTOOL = 0x2 SIDECAR_EXIFTOOL = 0x2
SIDECAR_XMP = 0x4 SIDECAR_XMP = 0x4
# supported attributes for --xattr-template
EXTENDED_ATTRIBUTE_NAMES = [
"authors",
"comment",
"copyright",
"description",
"findercomment",
"headline",
"keywords",
]
EXTENDED_ATTRIBUTE_NAMES_QUOTED = [f"'{x}'" for x in EXTENDED_ATTRIBUTE_NAMES]

View File

@@ -1,5 +1,5 @@
""" version info """ """ version info """
__version__ = "0.39.1" __version__ = "0.39.2"

View File

@@ -81,7 +81,7 @@ setup(
"wurlitzer>=2.0.1", "wurlitzer>=2.0.1",
"photoscript>=0.1.0", "photoscript>=0.1.0",
"toml>=0.10.0", "toml>=0.10.0",
"osxmetadata>=0.99.11", "osxmetadata>=0.99.13",
], ],
entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]}, entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]},
include_package_data=True, include_package_data=True,

File diff suppressed because one or more lines are too long

View File

@@ -5115,3 +5115,96 @@ def test_export_finder_tag_template_keywords():
persons = [persons] if type(persons) != list else persons persons = [persons] if type(persons) != list else persons
expected = [Tag(x) for x in keywords + persons] expected = [Tag(x) for x in keywords + persons]
assert sorted(md.tags) == sorted(expected) assert sorted(md.tags) == sorted(expected)
def test_export_xattr_template():
""" test --xattr template """
import glob
import os
import os.path
from osxmetadata import OSXMetaData
from osxphotos.__main__ import export
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
for uuid in CLI_FINDER_TAGS:
result = runner.invoke(
export,
[
os.path.join(cwd, PHOTOS_DB_15_7),
".",
"-V",
"--xattr-template",
"keywords",
"{person}",
"--xattr-template",
"comment",
"{title}",
"--uuid",
f"{uuid}",
],
)
assert result.exit_code == 0
md = OSXMetaData(CLI_FINDER_TAGS[uuid]["File:FileName"])
expected = CLI_FINDER_TAGS[uuid]["XMP:PersonInImage"]
expected = [expected] if type(expected) != list else expected
assert sorted(md.keywords) == sorted(expected)
assert md.comment == CLI_FINDER_TAGS[uuid]["XMP:Title"]
# run again with --update, should skip writing extended attributes
result = runner.invoke(
export,
[
os.path.join(cwd, PHOTOS_DB_15_7),
".",
"-V",
"--xattr-template",
"keywords",
"{person}",
"--xattr-template",
"comment",
"{title}",
"--uuid",
f"{uuid}",
"--update",
],
)
assert result.exit_code == 0
assert "Skipping extended attribute keywords" in result.output
assert "Skipping extended attribute comment" in result.output
# clear tags and run again, should update extended attributes
md.keywords = None
md.comment = None
result = runner.invoke(
export,
[
os.path.join(cwd, PHOTOS_DB_15_7),
".",
"-V",
"--xattr-template",
"keywords",
"{person}",
"--xattr-template",
"comment",
"{title}",
"--uuid",
f"{uuid}",
"--update",
],
)
assert result.exit_code == 0
assert "Writing extended attribute keyword" in result.output
assert "Writing extended attribute comment" in result.output
md = OSXMetaData(CLI_FINDER_TAGS[uuid]["File:FileName"])
expected = CLI_FINDER_TAGS[uuid]["XMP:PersonInImage"]
expected = [expected] if type(expected) != list else expected
assert sorted(md.keywords) == sorted(expected)
assert md.comment == CLI_FINDER_TAGS[uuid]["XMP:Title"]