forked/osxphotos
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Feature filter filter 759 (#771)

Browse Source 
* Added filter(x) filter, #759

* Added int, float filters
This commit is contained in:
Rhet Turnbull
2022-08-26 06:39:32 -07:00
committed by GitHub
parent d845e9b66e
commit 66f6002a57
3 changed files with 133 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
osxphotos/phototemplate.md
View File

@@ -42,7 +42,7 @@ Valid filters are:
- `parens`: Enclose value in parentheses, e.g. 'value' => '(value')
- `brackets`: Enclose value in brackets, e.g. 'value' => '[value]'
- `shell_quote`: Quotes the value for safe usage in the shell, e.g. My file.jpeg => 'My file.jpeg'; only adds quotes if needed.
- `function`: Run custom python function to filter value; use in format 'function:/path/to/file.py::function_name'. See example at https://github.com/RhetTbull/osxphotos/blob/master/examples/template_filter.py
- `function`: Run custom python function to filter value; use in format 'function:/path/to/file.py::function_name'. See example at <https://github.com/RhetTbull/osxphotos/blob/master/examples/template_filter.py>
- `split(x)`: Split value into a list of values using x as delimiter, e.g. 'value1;value2' => ['value1', 'value2'] if used with split(;).
- `autosplit`: Automatically split delimited string into separate values; will split strings delimited by comma, semicolon, or space, e.g. 'value1,value2' => ['value1', 'value2'].
- `chop(x)`: Remove x characters off the end of value, e.g. chop(1): 'Value' => 'Valu'; when applied to a list, chops characters from each list value, e.g. chop(1): ['travel', 'beach']=> ['trave', 'beac'].
@@ -57,6 +57,9 @@ Valid filters are:
- `remove(x)`: Remove x from list of values, e.g. remove(b): ['a', 'b', 'c'] => ['a', 'c'].
- `slice(start:stop:step)`: Slice list using same semantics as Python's list slicing, e.g. slice(1:3): ['a', 'b', 'c', 'd'] => ['b', 'c']; slice(1:4:2): ['a', 'b', 'c', 'd'] => ['b', 'd']; slice(1:): ['a', 'b', 'c', 'd'] => ['b', 'c', 'd']; slice(:-1): ['a', 'b', 'c', 'd'] => ['a', 'b', 'c']; slice(::-1): ['a', 'b', 'c', 'd'] => ['d', 'c', 'b', 'a']. See also sslice().
- `sslice(start:stop:step)`: [s(tring) slice] Slice values in a list using same semantics as Python's string slicing, e.g. sslice(1:3):'abcd => 'bc'; sslice(1:4:2): 'abcd' => 'bd', etc. See also slice().
- `filter(x)`: Filter list of values using predicate x; for example, `{folder_album|filter(contains Events)}` returns only folders/albums containing the word 'Events' in their path.
- int: Convert values in list to integer, e.g. 1.0 => 1. If value cannot be converted to integer, remove value from list. ['1.1', 'x'] => ['1']. See also float.
- float: Convert values in list to floating point number, e.g. 1 => 1.0. If value cannot be converted to float, remove value from list. ['1', 'x'] => ['1.0']. See also int.
e.g. if Photo keywords are `["FOO","bar"]`:
@@ -75,6 +78,21 @@ e.g. If Photo is in `Album1` in `Folder1`:
- `"{folder_album(>)}"` renders to `["Folder1>Album1"]`
- `"{folder_album()}"` renders to `["Folder1Album1"]`
The `filter(x)` filter uses the format `operator value` or `not operator value` to filter a list of values, returning only the values that match the predicate statement. The operators are the same as those used with the template conditional:
- `contains`: template field contains value, similar to python's `in`
- `matches`: template field contains exactly value, unlike `contains`: does not match partial matches
- `startswith`: template field starts with value
- `endswith`: template field ends with value
- `<=`: template field is less than or equal to value
- `>=`: template field is greater than or equal to value
- `<`: template field is less than value
- `>`: template field is greater than value
- `==`: template field equals value
- `!=`: template field does not equal value
The `filter(x)` filter is useful for excluding certain values, for example, if you have photos that are in multiple albums but you want to exclude certain albums from the list of exported directories when used with `--directory "{folder_album}"`. If a photo was in `Album1` and `Album2` and you wanted to exclude `Album2` from the list of output directories, you would use the filter: `--directory "{folder_album|filter(not contains Album2)}"`.
`[find,replace]`: optional text replacement to perform on rendered template value. For example, to replace "/" in an album name, you could use the template `"{album[/,-]}"`. Multiple replacements can be made by appending "|" and adding another find|replace pair. e.g. to replace both "/" and ":" in album name: `"{album[/,-|:,-]}"`. find/replace pairs are not limited to single characters. The "|" character cannot be used in a find/replace pair.
`conditional`: optional conditional expression that is evaluated as boolean (True/False) for use with the `?bool_value` modifier. Conditional expressions take the form '`not operator value`' where `not` is an optional modifier that negates the `operator`. Note: the space before the conditional expression is required if you use a conditional expression. Valid comparison operators are:
@@ -99,6 +117,9 @@ For example:
- `{photo.score.overall > 0.7}` resolves to True if the photo's overall aesthetic score is greater than 0.7.
- `{keyword|lower contains beach}` uses the lower case filter to do case-insensitive matching to match any keyword that contains the word 'beach'.
- `{keyword|lower not contains beach}` uses the `not` modifier to negate the comparison so this resolves to True if there is no keyword that matches 'beach'.
- `{keyword matches Beach|Travel}` resolves to True if 'Beach' or 'Travel' is a keyword. It would not match keyword 'BeachDay'.
Some templates like `{folder_album}` and `{keyword}` can return lists of values -- a photo can belong to more than one album or have more than one keyword. The `matches` operator returns True if any of the values in the list match the value. The `==` operator works on single values so it cannot be directly used on a list of values such as `{keywords}`. If you wanted to test that a photo has exactly a certain list of keywords, you can use filters to transform the values into a single value which can then be compared with the `==` operator. For example, if you wanted to test that a photo has exactly the keywords 'Beach' and 'Travel', you could use the template `"{keyword|sort|join == BeachTravel}"`. The `|sort|join` filter will sort the list of keywords and join them into a single string which can then be compared with `==`.
Examples: to export photos that contain certain keywords with the `osxphotos export` command's `--directory` option:
@@ -150,4 +171,4 @@ Variables can also be referenced as fields in the template string, for example:
If you need to use a `%` (percent sign character), you can escape the percent sign by using `%%`. You can also use the `{percent}` template field where a template field is required. For example:
`{title[:,%%]}` replaces the `:` with `%` and `{title contains Foo?{title}{percent},{title}}` adds `%` to the title if it contains `Foo`.
`{title[:,%%]}` replaces the `:` with `%` and `{title contains Foo?{title}{percent},{title}}` adds `%` to the title if it contains `Foo`.

102
osxphotos/phototemplate.py
View File

@@ -268,6 +268,11 @@ FILTER_VALUES = {
+ "slice(::-1): ['a', 'b', 'c', 'd'] => ['d', 'c', 'b', 'a']. See also sslice().",
"sslice(start:stop:step)": "[s(tring) slice] Slice values in a list using same semantics as Python's string slicing, "
+ "e.g. sslice(1:3):'abcd => 'bc'; sslice(1:4:2): 'abcd' => 'bd', etc. See also slice().",
"filter(x)": "Filter list of values using predicate x; for example, `{folder_album|filter(contains Events)}` returns only folders/albums containing the word 'Events' in their path.",
"int": "Convert values in list to integer, e.g. 1.0 => 1. If value cannot be converted to integer, remove value from list. "
+ "['1.1', 'x'] => ['1']. See also float.",
"float": "Convert values in list to floating point number, e.g. 1 => 1.0. If value cannot be converted to float, remove value from list. "
+ "['1', 'x'] => ['1.0']. See also int.",
}
# Just the substitutions without the braces
@@ -667,16 +672,18 @@ class PhotoTemplate:
vals = string_test(lambda v, c: v.endswith(c))
elif operator == "==":
match = sorted(vals) == sorted(conditional_value)
if (match and not negation) or (negation and not match):
vals = ["True"]
else:
vals = []
vals = (
["True"]
if (match and not negation) or (negation and not match)
else []
)
elif operator == "!=":
match = sorted(vals) != sorted(conditional_value)
if (match and not negation) or (negation and not match):
vals = ["True"]
else:
vals = []
vals = (
["True"]
if (match and not negation) or (negation and not match)
else []
)
elif operator == "<":
vals = comparison_test(lambda v, c: v < c)
elif operator == "<=":
@@ -1182,6 +1189,7 @@ class PhotoTemplate:
"remove",
"slice",
"sslice",
"filter",
] and (args is None or not len(args)):
raise SyntaxError(f"{filter_} requires arguments")
@@ -1270,12 +1278,72 @@ class PhotoTemplate:
# slice each value in a list
slice_ = create_slice(args)
value = [v[slice_] for v in values]
elif filter_ == "filter":
# filter values based on a predicate
value = [v for v in values if self.filter_predicate(v, args)]
elif filter_ == "int":
# convert value to integer
value = values_to_int(values)
elif filter_ == "float":
# convert value to float
value = values_to_float(values)
elif filter_.startswith("function:"):
value = self.get_template_value_filter_function(filter_, args, values)
else:
value = []
return value
def filter_predicate(self, value: str, args: str) -> bool:
"""Return True if value passes predicate"""
# extract function name and arguments
if not args:
raise SyntaxError("Filter predicate requires arguments")
args = args.split(None, 1)
if args[0] == "not":
args = args[1:]
return not self.filter_predicate(value, " ".join(args))
predicate = args[0]
conditional_value = args[1].split("|")
def comparison_test(test_function):
"""Perform numerical comparisons using test_function"""
# returns True if any of the values match the condition
try:
return any(
bool(test_function(float(value), float(c)))
for c in conditional_value
)
except ValueError as e:
raise SyntaxError(
f"comparison operators may only be used with values that can be converted to numbers: {vals} {conditional_value}"
) from e
predicate_is_true = False
if predicate == "contains":
predicate_is_true = any(c in value for c in conditional_value)
elif predicate == "endswith":
predicate_is_true = any(value.endswith(c) for c in conditional_value)
elif predicate in ["matches", "=="]:
predicate_is_true = any(value == c for c in conditional_value)
elif predicate == "startswith":
predicate_is_true = any(value.startswith(c) for c in conditional_value)
elif predicate == "!=":
predicate_is_true = any(value != c for c in conditional_value)
elif predicate == "<":
predicate_is_true = comparison_test(lambda v, c: v < c)
elif predicate == "<=":
predicate_is_true = comparison_test(lambda v, c: v <= c)
elif predicate == ">":
predicate_is_true = comparison_test(lambda v, c: v > c)
elif predicate == ">=":
predicate_is_true = comparison_test(lambda v, c: v >= c)
else:
raise SyntaxError(f"Invalid predicate: {predicate}")
return predicate_is_true
def get_template_value_multi(self, field, subfield, path_sep, default):
"""lookup value for template field (multi-value template substitutions)
@@ -1730,3 +1798,21 @@ def create_slice(args):
else:
raise SyntaxError(f"Invalid slice: {args}")
return slice(start, end, step)
def values_to_int(values: List[str]) -> List[str]:
"""Convert a list of strings to str representation of ints, if possible, otherwise strip values from list"""
int_values = []
for v in values:
with suppress(ValueError):
int_values.append(str(int(float(v))))
return int_values
def values_to_float(values: List[str]) -> List[str]:
"""Convert a list of strings to str representation of float, if possible, otherwise strip values from list"""
float_values = []
for v in values:
with suppress(ValueError):
float_values.append(str(float(v)))
return float_values

16
tests/test_template.py
View File

@@ -238,6 +238,21 @@ TEMPLATE_VALUES = {
"{descr|autosplit|slice(:2)|join()}": "JackRose",
"{descr|autosplit|slice(:-1)|join()}": "JackRoseDining",
"{descr|autosplit|slice(::2)|join()}": "JackDining",
"{descr|filter(startswith Jack)}": "Jack Rose Dining Saloon",
"{descr|filter(startswith Rose)}": "_",
"{descr|filter(endswith Saloon)}": "Jack Rose Dining Saloon",
"{descr|filter(endswith Rose)}": "_",
"{descr|filter(contains Rose)}": "Jack Rose Dining Saloon",
"{descr|filter(not contains Rose)}": "_",
"{descr|filter(matches Jack Rose Dining Saloon)}": "Jack Rose Dining Saloon",
"{created.mm|filter(== 02)}": "02",
"{created.mm|filter(<= 2)}": "02",
"{created.mm|filter(>= 2)}": "02",
"{created.mm|filter(> 3)}": "_",
"{created.mm|filter(< 1)}": "_",
"{created.mm|filter(!= 02)}": "_",
"{created.mm|int|filter(== 2)}": "2",
"{created.mm|float|filter(== 2.0)}": "2.0",
}
@@ -437,6 +452,7 @@ UUID_ALBUM_SEQ = {
"{folder_album_seq(1)}": "2",
"{folder_album_seq(0)}": "1",
"{folder_album_seq:03d(1)}": "002",
"{folder_album|filter(startswith Folder1)}": "Folder1/SubFolder2/AlbumInFolder",
},
},
}