forked/osxphotos
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Initial version of templating system for CLI

Browse Source
This commit is contained in:
Rhet Turnbull 2020-03-22 09:45:56 -07:00
parent d26ea0dccc
commit 2feb0999b3
7 changed files with 500 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

81
README.md
View File

@ -213,19 +213,100 @@ Options:
exiftool must be installed and in the path.
exiftool may be installed from
https://exiftool.org/
--directory DIRECTORY Optional template for specifying name of
output directory. See below for additional
details on templating system
-h, --help Show this message and exit.
**Templating System**
With the --directory option, you may specify a template for the export
directory. This directory will be appended to the export path specified in
the export DEST argument to export. For example, if template is
'{created.year}/{created.month}', and export desitnation DEST is
'/Users/maria/Pictures/export', the actual export directory for a photo would
be '/Users/maria/Pictures/export/2020/March' if the photo was created in
March 2020.
In the template, valid template substitutions will be replaced by the
corresponding value from the table below. Invalid substitutions will result
in a warning but will be left unchanged. e.g. if you put '{foo}' in your
template, e.g. '{created.year}/{foo}', the resulting output directory would
look like '/Users/maria/Pictures/export/2020/{foo}'
If you want the actual text of the template substition to appear in the
rendered name, escape the curly braces with \, for example, using
'{created.year}/\{name\}' for --directory would result in output of
2020/{name}/photoname.jpg
In the current implementation, substitutions which have no value will be
replaced by '_', for example, your template looked like
'{created.year}/{place.address}' but there was no address associated with the
photo, the resulting output would be: '2020/_/photoname.jpg'
I plan to add the option to specify the value to be used for missing
subsitutions in a future version. I also plan to extend the templating system
to the exported filename so you can specify the filename using a template.
Substitution Description
{name} Filename of the photo
{original_name} Photo's original filename when imported to Photos
{title} Title of the photo
{descr} Description of the photo
{created.date} Photo's creation date in ISO format, e.g. '2020-03-22'
{created.year} 4-digit year of file creation time
{created.yy} 2-digit year of file creation time
{created.mm} 2-digit month of the file creation time (zero padded)
{created.month} Month name in user's locale of the file creation time
{created.mon} Month abbreviation in the user's locale of the file
creation time
{created.doy} 3-digit day of year (e.g Julian day) of file creation
time, starting from 1 (zero padded)
{modified.date} Photo's modification date in ISO format, e.g.
'2020-03-22'
{modified.year} 4-digit year of file modification time
{modified.yy} 2-digit year of file modification time
{modified.mm} 2-digit month of the file modification time (zero
padded)
{modified.month} Month name in user's locale of the file modification
time
{modified.mon} Month abbreviation in the user's locale of the file
modification time
{modified.doy} 3-digit day of year (e.g Julian day) of file
modification time, starting from 1 (zero padded)
{place.name} Place name from the photo's reverse geolocation data
{place.names} list of place names from the photo's reverse
geolocation data, joined with '_', for example, '18th
St NW_Washington_DC_United States'
{place.address} Postal address from the photo's reverse geolocation
data, e.g. '2007 18th St NW, Washington, DC 20009,
United States'
{place.street} Street part of the postal address, e.g. '2007 18th St
NW'
{place.city} City part of the postal address, e.g. 'Washington'
{place.state} State part of the postal address, e.g. 'DC'
{place.postal_code} Postal code part of the postal address, e.g. '20009'
{place.country} Country name of the postal code, e.g. 'United States'
{place.country_code} ISO country code of the postal address, e.g. 'US'
```
Example: export all photos to ~/Desktop/export, including edited versions and live photo movies, group in folders by date created
`osxphotos export --export-edited --export-live --export-by-date ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
**Note**: Photos library/database path can also be specified using --db option:
`osxphotos export --export-edited --export-live --export-by-date --db ~/Pictures/Photos\ Library.photoslibrary ~/Desktop/export`
Example: find all photos with keyword "Kids" and output results to json file named results.json:
`osxphotos query --keyword Kids --json ~/Pictures/Photos\ Library.photoslibrary >results.json`
Example: export photos to file structure based on 4-digit year and full name of month of photo's creation date:
`osxphotos export ~/Desktop/export --directory "{created.year}/{created.month}"`
## Example uses of the package
```python

97
osxphotos/__main__.py
View File

@ -1,6 +1,7 @@
import csv
import datetime
import json
import logging
import os
import os.path
import pathlib
@ -8,13 +9,20 @@ import sys
import click
import yaml
from pathvalidate import (
is_valid_filename,
is_valid_filepath,
sanitize_filepath,
sanitize_filename,
)
import osxphotos
from ._constants import _EXIF_TOOL_URL, _PHOTOS_5_VERSION
from ._version import __version__
from .utils import create_path_by_date, _copy_file
from .exiftool import get_exiftool_path
from .template import render_filename_template, TEMPLATE_SUBSTITUTIONS
from .utils import _copy_file, create_path_by_date
def get_photos_db(*db_options):
@ -59,6 +67,65 @@ class CLI_Obj:
self.json = json
class ExportCommand(click.Command):
""" Custom click.Command that overrides get_help() to show additional help info for export """
def get_help(self, ctx):
help_text = super().get_help(ctx)
formatter = click.HelpFormatter()
formatter.write("\n\n")
# passed to click.HelpFormatter.write_dl for formatting
formatter.write_text("**Templating System**")
formatter.write("\n")
formatter.write_text(
"With the --directory option, you may specify a template for the "
+ "export directory. This directory will be appended to the export path specified "
+ " in the export DEST argument to export. For example, if template is "
+ "'{created.year}/{created.month}', and export desitnation DEST is "
+ "'/Users/maria/Pictures/export', "
+ " the actual export directory for a photo would be '/Users/maria/Pictures/export/2020/March' "
+ " if the photo was created in March 2020. "
)
formatter.write("\n")
formatter.write_text(
"In the template, valid template substitutions will be replaced by "
+ "the corresponding value from the table below. Invalid substitutions will result in a "
+ "warning but will be left unchanged. e.g. if you put '{foo}' in your template, "
+ "e.g. '{created.year}/{foo}', the resulting output directory would look like "
+ "'/Users/maria/Pictures/export/2020/{foo}' "
)
formatter.write("\n")
formatter.write_text(
"If you want the actual text of the template substition to appear "
+ "in the rendered name, escape the curly braces with \\, for example, "
+ "using '{created.year}/\\{name\\}' for --directory "
+ "would result in output of 2020/{name}/photoname.jpg"
)
formatter.write("\n")
formatter.write_text(
"In the current implementation, substitutions which have no value "
+ "will be replaced by '_', "
+ "for example, your template looked like '{created.year}/{place.address}' "
+ "but there was no address associated with the photo, the resulting output would be: "
+ "'2020/_/photoname.jpg' "
)
formatter.write("\n")
formatter.write_text(
"I plan to add the option to specify the value to be used for missing "
+ "subsitutions in a future version. I also plan to extend the templating system "
+ "to the exported filename so you can specify the filename using a template."
)
formatter.write("\n")
templ_tuples = [("Substitution", "Description")]
templ_tuples.extend((k, v) for k, v in TEMPLATE_SUBSTITUTIONS.items())
formatter.write_dl(templ_tuples)
help_text += formatter.getvalue()
return help_text
CTX_SETTINGS = dict(help_option_names=["-h", "--help"])
DB_OPTION = click.option(
"--db",
@ -674,7 +741,7 @@ def query(
print_photo_info(photos, cli_json or json_)
@cli.command()
@cli.command(cls=ExportCommand)
@DB_OPTION
@query_options
@click.option("--verbose", "-V", is_flag=True, help="Print verbose output.")
@ -746,6 +813,13 @@ def query(
"To use this option, exiftool must be installed and in the path. "
"exiftool may be installed from https://exiftool.org/",
)
@click.option(
"--directory",
metavar="DIRECTORY",
default=None,
help="Optional template for specifying name of output directory. "
"See below for additional details on templating system",
)
@DB_ARGUMENT
@click.argument("dest", nargs=1, type=click.Path(exists=True))
@click.pass_obj
@ -806,6 +880,7 @@ def export(
not_selfie,
panorama,
not_panorama,
directory,
):
""" Export photos from the Photos database.
Export path DEST is required.
@ -834,6 +909,7 @@ def export(
(hdr, not_hdr),
(selfie, not_selfie),
(panorama, not_panorama),
(export_by_date, directory),
]
if any([all(bb) for bb in exclusive]):
click.echo(cli.commands["export"].get_help(ctx), err=True)
@ -943,6 +1019,7 @@ def export(
export_live,
download_missing,
exiftool,
directory,
)
else:
for p in photos:
@ -958,6 +1035,7 @@ def export(
export_live,
download_missing,
exiftool,
directory,
)
if export_path:
click.echo(f"Exported {p.filename} to {export_path}")
@ -1276,6 +1354,7 @@ def export_photo(
export_live,
download_missing,
exiftool,
directory,
):
""" Helper function for export that does the actual export
photo: PhotoInfo object
@ -1289,6 +1368,7 @@ def export_photo(
live video will have same name as photo but with .mov extension
download_missing: attempt download of missing iCloud photos
exiftool: use exiftool to write EXIF metadata directly to exported photo
directory: template used to determine output directory
returns destination path of exported photo or None if photo was missing
"""
@ -1322,6 +1402,18 @@ def export_photo(
if export_by_date:
date_created = photo.date.timetuple()
dest = create_path_by_date(dest, date_created)
elif directory:
dirname, unmatched = render_filename_template(directory, photo)
if unmatched:
click.echo(
f"Possible unmatched substitution in template: {unmatched}", err=True
)
dirname = sanitize_filepath(dirname)
if not is_valid_filepath(dirname):
raise ValueError(f"Invalid file path: {dirname}")
dest = os.path.join(dest, dirname)
if not os.path.isdir(dest):
os.makedirs(dest)
sidecar = [s.lower() for s in sidecar]
sidecar_json = sidecar_xmp = False
@ -1382,4 +1474,3 @@ def export_photo(
if __name__ == "__main__":
cli() # pylint: disable=no-value-for-parameter

2
osxphotos/_version.py
View File

@ -1,3 +1,3 @@
""" version info """
__version__ = "0.23.2"
__version__ = "0.23.3"

200
osxphotos/template.py Normal file
View File

@ -0,0 +1,200 @@
import datetime
import pathlib
import re
from typing import Tuple # pylint: disable=syntax-error
from .photoinfo import PhotoInfo
TEMPLATE_SUBSTITUTIONS = {
"{name}": "Filename of the photo",
"{original_name}": "Photo's original filename when imported to Photos",
"{title}": "Title of the photo",
"{descr}": "Description of the photo",
"{created.date}": "Photo's creation date in ISO format, e.g. '2020-03-22'",
"{created.year}": "4-digit year of file creation time",
"{created.yy}": "2-digit year of file creation time",
"{created.mm}": "2-digit month of the file creation time (zero padded)",
"{created.month}": "Month name in user's locale of the file creation time",
"{created.mon}": "Month abbreviation in the user's locale of the file creation time",
"{created.doy}": "3-digit day of year (e.g Julian day) of file creation time, starting from 1 (zero padded)",
"{modified.date}": "Photo's modification date in ISO format, e.g. '2020-03-22'",
"{modified.year}": "4-digit year of file modification time",
"{modified.yy}": "2-digit year of file modification time",
"{modified.mm}": "2-digit month of the file modification time (zero padded)",
"{modified.month}": "Month name in user's locale of the file modification time",
"{modified.mon}": "Month abbreviation in the user's locale of the file modification time",
"{modified.doy}": "3-digit day of year (e.g Julian day) of file modification time, starting from 1 (zero padded)",
"{place.name}": "Place name from the photo's reverse geolocation data",
"{place.names}": "list of place names from the photo's reverse geolocation data, joined with '_', for example, '18th St NW_Washington_DC_United States'",
"{place.address}": "Postal address from the photo's reverse geolocation data, e.g. '2007 18th St NW, Washington, DC 20009, United States'",
"{place.street}": "Street part of the postal address, e.g. '2007 18th St NW'",
"{place.city}": "City part of the postal address, e.g. 'Washington'",
"{place.state}": "State part of the postal address, e.g. 'DC'",
"{place.postal_code}": "Postal code part of the postal address, e.g. '20009'",
"{place.country}": "Country name of the postal code, e.g. 'United States'",
"{place.country_code}": "ISO country code of the postal address, e.g. 'US'",
}
def render_filename_template(
template: str, photo: PhotoInfo, none_str: str = "_"
) -> Tuple[str, list]:
""" render a filename or directory template """
if type(template) is not str:
raise TypeError(f"template must be type str, not {type(template)}")
if type(photo) is not PhotoInfo:
raise TypeError(f"photo must be type osxphotos.PhotoInfo, not {type(photo)}")
rendered = template
original_name = pathlib.Path(photo.original_filename).stem
current_name = pathlib.Path(photo.filename).stem
created = DateTimeFormatter(photo.date)
if photo.date_modified:
modified = DateTimeFormatter(photo.date_modified)
else:
modified = None
# make substitutions
rendered = rendered.replace("{name}", current_name)
rendered = rendered.replace("{original_name}", original_name)
title = photo.title if photo.title is not None else none_str
rendered = rendered.replace("{title}", f"{title}")
descr = photo.description if photo.description is not None else none_str
rendered = rendered.replace("{descr}", f"{descr}")
rendered = rendered.replace("{created.date}", photo.date.date().isoformat())
rendered = rendered.replace("{created.year}", created.year)
rendered = rendered.replace("{created.yy}", created.yy)
rendered = rendered.replace("{created.mm}", created.mm)
rendered = rendered.replace("{created.month}", created.month)
rendered = rendered.replace("{created.mon}", created.mon)
rendered = rendered.replace("{created.doy}", created.doy)
if modified is not None:
rendered = rendered.replace(
"{modified.date}", photo.date_modified.date().isoformat()
)
rendered = rendered.replace("{modified.year}", modified.year)
rendered = rendered.replace("{modified.yy}", modified.yy)
rendered = rendered.replace("{modified.mm}", modified.mm)
rendered = rendered.replace("{modified.month}", modified.month)
rendered = rendered.replace("{modified.mon}", modified.mon)
rendered = rendered.replace("{modified.doy}", modified.doy)
else:
rendered = rendered.replace("{modified.year}", none_str)
rendered = rendered.replace("{modified.yy}", none_str)
rendered = rendered.replace("{modified.mm}", none_str)
rendered = rendered.replace("{modified.month}", none_str)
rendered = rendered.replace("{modified.mon}", none_str)
rendered = rendered.replace("{modified.doy}", none_str)
place_name = photo.place.name if photo.place and photo.place.name else none_str
rendered = rendered.replace("{place.name}", place_name)
place_names = (
"_".join(photo.place.names) if photo.place and photo.place.names else none_str
)
rendered = rendered.replace("{place.names}", place_names)
address = (
photo.place.address_str if photo.place and photo.place.address_str else none_str
)
rendered = rendered.replace("{place.address}", address)
street = (
photo.place.address.street
if photo.place and photo.place.address.street
else none_str
)
rendered = rendered.replace("{place.street}", street)
city = (
photo.place.address.city
if photo.place and photo.place.address.city
else none_str
)
rendered = rendered.replace("{place.city}", city)
state = (
photo.place.address.state
if photo.place and photo.place.address.state
else none_str
)
rendered = rendered.replace("{place.state}", state)
postal_code = (
photo.place.address.state
if photo.place and photo.place.address.postal_code
else none_str
)
rendered = rendered.replace("{place.postal_code}", postal_code)
country = (
photo.place.address.state
if photo.place and photo.place.address.country
else none_str
)
rendered = rendered.replace("{place.country}", country)
country_code = (
photo.place.country_code
if photo.place and photo.place.country_code
else none_str
)
rendered = rendered.replace("{place.country_code}", country_code)
# fix any escaped curly braces
rendered = re.sub(r"\\{", "{", rendered)
rendered = re.sub(r"\\}", "}", rendered)
# find any {words} that weren't replaced
unmatched = re.findall(r"{\w+}", rendered)
return (rendered, unmatched)
class DateTimeFormatter:
""" provides property access to formatted datetime.datetime strftime values """
def __init__(self, dt: datetime.datetime):
self.dt = dt
@property
def year(self):
""" 4 digit year """
year = f"{self.dt.year}"
return year
@property
def yy(self):
""" 2 digit year """
yy = f"{self.dt.strftime('%y')}"
return yy
@property
def mm(self):
""" 2 digit month """
mm = f"{self.dt.strftime('%m')}"
return mm
@property
def month(self):
""" Month as locale's full name """
month = f"{self.dt.strftime('%B')}"
return month
@property
def mon(self):
""" Month as locale's abbreviated name """
mon = f"{self.dt.strftime('%b')}"
return mon
@property
def doy(self):
""" Julian day of year starting from 001 """
doy = f"{self.dt.strftime('%j')}"
return doy

1
requirements.txt
View File

@ -24,6 +24,7 @@ modulegraph==0.18
more-itertools==7.2.0
packaging==19.0
pathspec==0.7.0
pathvalidate==2.2.1
pluggy==0.12.0
py==1.8.0
py2app==0.21

1
setup.py
View File

@ -68,6 +68,7 @@ setup(
"Mako>=1.1.1",
"bpylist2==2.0.3;python_version<'3.8'",
"bpylist2==3.0.0;python_version>='3.8'",
"pathvalidate==2.2.1",
],
entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]},
include_package_data=True,

122
tests/test_cli.py
View File

@ -39,6 +39,33 @@ CLI_EXPORT_FILENAMES = [
"wedding_edited.jpeg",
]
CLI_EXPORTED_DIRECTORY_TEMPLATE_FILENAMES1 = [
"2019/April/wedding.jpg",
"2019/July/Tulips.jpg",
"2018/October/St James Park.jpg",
"2018/September/Pumpkins3.jpg",
"2018/September/Pumkins2.jpg",
"2018/September/Pumkins1.jpg",
]
CLI_EXPORTED_DIRECTORY_TEMPLATE_FILENAMES2 = [
"St James's Park/St James Park.jpg",
"_/Pumpkins3.jpg",
"_/Pumkins2.jpg",
"_/Pumkins1.jpg",
"_/Tulips.jpg",
"_/wedding.jpg",
]
CLI_EXPORTED_DIRECTORY_TEMPLATE_FILENAMES3 = [
"2019/{foo}/wedding.jpg",
"2019/{foo}/Tulips.jpg",
"2018/{foo}/St James Park.jpg",
"2018/{foo}/Pumpkins3.jpg",
"2018/{foo}/Pumkins2.jpg",
"2018/{foo}/Pumkins1.jpg",
]
CLI_EXPORT_UUID = "D79B8D77-BFFC-460B-9312-034F2877D35B"
CLI_EXPORT_SIDECAR_FILENAMES = ["Pumkins2.jpg", "Pumkins2.json", "Pumkins2.xmp"]
@ -117,6 +144,7 @@ def test_export():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export,
@ -171,6 +199,7 @@ def test_export_sidecar():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
cli,
@ -199,6 +228,7 @@ def test_export_live():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export,
@ -224,6 +254,7 @@ def test_export_raw():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(export, [os.path.join(cwd, RAW_PHOTOS_DB), ".", "-V"])
files = glob.glob("*")
@ -239,6 +270,7 @@ def test_export_raw_original():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export, [os.path.join(cwd, RAW_PHOTOS_DB), ".", "--original-name", "-V"]
@ -256,6 +288,7 @@ def test_export_raw_edited():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export, [os.path.join(cwd, RAW_PHOTOS_DB), ".", "--export-edited", "-V"]
@ -273,6 +306,7 @@ def test_export_raw_edited_original():
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export,
@ -286,3 +320,91 @@ def test_export_raw_edited_original():
)
files = glob.glob("*")
assert sorted(files) == sorted(CLI_EXPORT_RAW_EDITED_ORIGINAL)
def test_export_directory_template_1():
# test export using directory template
import glob
import os
import os.path
import osxphotos
from osxphotos.__main__ import export
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export,
[
os.path.join(cwd, CLI_PHOTOS_DB),
".",
"--original-name",
"-V",
"--directory",
"{created.year}/{created.month}",
],
)
assert result.exit_code == 0
workdir = os.getcwd()
for filepath in CLI_EXPORTED_DIRECTORY_TEMPLATE_FILENAMES1:
assert os.path.isfile(os.path.join(workdir, filepath))
def test_export_directory_template_2():
# test export using directory template with missing substitution value
import glob
import os
import os.path
import osxphotos
from osxphotos.__main__ import export
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export,
[
os.path.join(cwd, CLI_PHOTOS_DB),
".",
"--original-name",
"-V",
"--directory",
"{place.name}",
],
)
assert result.exit_code == 0
workdir = os.getcwd()
for filepath in CLI_EXPORTED_DIRECTORY_TEMPLATE_FILENAMES2:
assert os.path.isfile(os.path.join(workdir, filepath))
def test_export_directory_template_3():
# test export using directory template with unmatched substituion value
import glob
import os
import os.path
import osxphotos
from osxphotos.__main__ import export
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
result = runner.invoke(
export,
[
os.path.join(cwd, CLI_PHOTOS_DB),
".",
"--original-name",
"-V",
"--directory",
"{created.year}/{foo}",
],
)
assert result.exit_code == 0
assert "Possible unmatched substitution in template: ['{foo}']" in result.output
workdir = os.getcwd()
for filepath in CLI_EXPORTED_DIRECTORY_TEMPLATE_FILENAMES3:
assert os.path.isfile(os.path.join(workdir, filepath))