forked/osxphotos
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Concurrency refactor 999 (#1029)

Browse Source 
* Working on making export threadsafe, #999

* Working on making export threadsafe, #999

* refactor for concurrent export, #999

* Fixed race condition in ExportRecord context manager
This commit is contained in:
Rhet Turnbull 2023-04-01 09:39:08 -07:00 committed by GitHub
parent 2c4d0f4546
commit e7099d250b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
18 changed files with 1453 additions and 676 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

156
API_README.md
View File

@ -1523,12 +1523,14 @@ Returns full name of the album owner (person who shared the album) for shared al
**Note**: *Only valid on Photos 5 / MacOS 10.15+; on Photos <= 4, returns None.
#### `asdict()`
Returns a dictionary representation of the AlbumInfo object.
### ImportInfo
PhotosDB.import_info returns a list of ImportInfo objects. Each ImportInfo object represents an import session in the library. PhotoInfo.import_info returns a single ImportInfo object representing the import session for the photo (or `None` if no associated import session).
**Note**: Photos 5+ only. Not implemented for Photos version <= 4.
#### `uuid`
Returns the universally unique identifier (uuid) of the import session. This is how Photos keeps track of individual objects within the database.
@ -1543,12 +1545,18 @@ Returns the creation date as a timezone aware datetime.datetime object of the im
#### `start_date`
Returns the start date as a timezone aware datetime.datetime object for when the import session bega.
Returns the start date as a timezone aware datetime.datetime object for when the import session began.
#### `end_date`
Returns the end date as a timezone aware datetime.datetime object for when the import session completed.
**Note**: On Photos <=4, `start_date` and `end_date` will be the same as `creation_date`.
#### `asdict()`
Returns a dictionary representation of the import session.
### ProjectInfo
PhotosDB.projcet_info returns a list of ProjectInfo objects. Each ProjectInfo object represents a project in the library. PhotoInfo.project_info returns a list of ProjectInfo objects for each project the photo is contained in.
@ -1571,6 +1579,10 @@ Returns a list of [PhotoInfo](#photoinfo) objects representing each photo contai
Returns the creation date as a timezone aware datetime.datetime object of the project.
#### `asdict()`
Returns a dictionary representation of the ProjectInfo object.
### MomentInfo
PhotoInfo.moment_info return the MomentInfo object for the photo. The MomentInfo object contains information about the photo's moment as assigned by Photos. The MomentInfo object contains the following properties:
@ -1661,6 +1673,10 @@ Returns album sort order (as `AlbumSortOrder` enum). On Photos <=4, always retu
Returns index of photo in album (based on album sort order).
#### `asdict()`
Returns a dictionary representation of the FolderInfo object.
**Note**: FolderInfo and AlbumInfo objects effectively work as a linked list. The children of a folder are contained in `subfolders` and `album_info` and the parent object of both `AlbumInfo` and `FolderInfo` is represented by `parent`. For example:
```pycon
@ -2134,10 +2150,10 @@ Template statements are white-space sensitive meaning that white space (spaces,
e.g. if Photo keywords are `["foo","bar"]`:
- `"{keyword}"` renders to `"foo", "bar"`
- `"{,+keyword}"` renders to: `"foo,bar"`
- `"{; +keyword}"` renders to: `"foo; bar"`
- `"{+keyword}"` renders to `"foobar"`
* `"{keyword}"` renders to `"foo", "bar"`
* `"{,+keyword}"` renders to: `"foo,bar"`
* `"{; +keyword}"` renders to: `"foo; bar"`
* `"{+keyword}"` renders to `"foobar"`
`template_field`: The template field to resolve. See [Template Substitutions](#template-substitutions) for full list of template fields.
@ -2149,77 +2165,77 @@ e.g. if Photo keywords are `["foo","bar"]`:
Valid filters are:
- `lower`: Convert value to lower case, e.g. 'Value' => 'value'.
- `upper`: Convert value to upper case, e.g. 'Value' => 'VALUE'.
- `strip`: Strip whitespace from beginning/end of value, e.g. ' Value ' => 'Value'.
- `titlecase`: Convert value to title case, e.g. 'my value' => 'My Value'.
- `capitalize`: Capitalize first word of value and convert other words to lower case, e.g. 'MY VALUE' => 'My value'.
- `braces`: Enclose value in curly braces, e.g. 'value => '{value}'.
- `parens`: Enclose value in parentheses, e.g. 'value' => '(value')
- `brackets`: Enclose value in brackets, e.g. 'value' => '[value]'
- `shell_quote`: Quotes the value for safe usage in the shell, e.g. My file.jpeg => 'My file.jpeg'; only adds quotes if needed.
- `function`: Run custom python function to filter value; use in format 'function:/path/to/file.py::function_name'. See example at https://github.com/RhetTbull/osxphotos/blob/master/examples/template_filter.py
- `split(x)`: Split value into a list of values using x as delimiter, e.g. 'value1;value2' => ['value1', 'value2'] if used with split(;).
- `autosplit`: Automatically split delimited string into separate values; will split strings delimited by comma, semicolon, or space, e.g. 'value1,value2' => ['value1', 'value2'].
- `chop(x)`: Remove x characters off the end of value, e.g. chop(1): 'Value' => 'Valu'; when applied to a list, chops characters from each list value, e.g. chop(1): ['travel', 'beach']=> ['trave', 'beac'].
- `chomp(x)`: Remove x characters from the beginning of value, e.g. chomp(1): ['Value'] => ['alue']; when applied to a list, removes characters from each list value, e.g. chomp(1): ['travel', 'beach']=> ['ravel', 'each'].
- `sort`: Sort list of values, e.g. ['c', 'b', 'a'] => ['a', 'b', 'c'].
- `rsort`: Sort list of values in reverse order, e.g. ['a', 'b', 'c'] => ['c', 'b', 'a'].
- `reverse`: Reverse order of values, e.g. ['a', 'b', 'c'] => ['c', 'b', 'a'].
- `uniq`: Remove duplicate values, e.g. ['a', 'b', 'c', 'b', 'a'] => ['a', 'b', 'c'].
- `join(x)`: Join list of values with delimiter x, e.g. join(,): ['a', 'b', 'c'] => 'a,b,c'; the DELIM option functions similar to join(x) but with DELIM, the join happens before being passed to any filters.May optionally be used without an argument, that is 'join()' which joins values together with no delimiter. e.g. join(): ['a', 'b', 'c'] => 'abc'.
- `append(x)`: Append x to list of values, e.g. append(d): ['a', 'b', 'c'] => ['a', 'b', 'c', 'd'].
- `prepend(x)`: Prepend x to list of values, e.g. prepend(d): ['a', 'b', 'c'] => ['d', 'a', 'b', 'c'].
- `appends(x)`: Append s[tring] Append x to each value of list of values, e.g. appends(d): ['a', 'b', 'c'] => ['ad', 'bd', 'cd'].
- `prepends(x)`: Prepend s[tring] x to each value of list of values, e.g. prepends(d): ['a', 'b', 'c'] => ['da', 'db', 'dc'].
- `remove(x)`: Remove x from list of values, e.g. remove(b): ['a', 'b', 'c'] => ['a', 'c'].
- `slice(start:stop:step)`: Slice list using same semantics as Python's list slicing, e.g. slice(1:3): ['a', 'b', 'c', 'd'] => ['b', 'c']; slice(1:4:2): ['a', 'b', 'c', 'd'] => ['b', 'd']; slice(1:): ['a', 'b', 'c', 'd'] => ['b', 'c', 'd']; slice(:-1): ['a', 'b', 'c', 'd'] => ['a', 'b', 'c']; slice(::-1): ['a', 'b', 'c', 'd'] => ['d', 'c', 'b', 'a']. See also sslice().
- `sslice(start:stop:step)`: [s(tring) slice] Slice values in a list using same semantics as Python's string slicing, e.g. sslice(1:3):'abcd => 'bc'; sslice(1:4:2): 'abcd' => 'bd', etc. See also slice().
- `filter(x)`: Filter list of values using predicate x; for example, `{folder_album|filter(contains Events)}` returns only folders/albums containing the word 'Events' in their path.
- `int`: Convert values in list to integer, e.g. 1.0 => 1. If value cannot be converted to integer, remove value from list. ['1.1', 'x'] => ['1']. See also float.
- `float`: Convert values in list to floating point number, e.g. 1 => 1.0. If value cannot be converted to float, remove value from list. ['1', 'x'] => ['1.0']. See also int.
* `lower`: Convert value to lower case, e.g. 'Value' => 'value'.
* `upper`: Convert value to upper case, e.g. 'Value' => 'VALUE'.
* `strip`: Strip whitespace from beginning/end of value, e.g. ' Value ' => 'Value'.
* `titlecase`: Convert value to title case, e.g. 'my value' => 'My Value'.
* `capitalize`: Capitalize first word of value and convert other words to lower case, e.g. 'MY VALUE' => 'My value'.
* `braces`: Enclose value in curly braces, e.g. 'value => '{value}'.
* `parens`: Enclose value in parentheses, e.g. 'value' => '(value')
* `brackets`: Enclose value in brackets, e.g. 'value' => '[value]'
* `shell_quote`: Quotes the value for safe usage in the shell, e.g. My file.jpeg => 'My file.jpeg'; only adds quotes if needed.
* `function`: Run custom python function to filter value; use in format 'function:/path/to/file.py::function_name'. See example at <https://github.com/RhetTbull/osxphotos/blob/master/examples/template_filter.py>
* `split(x)`: Split value into a list of values using x as delimiter, e.g. 'value1;value2' => ['value1', 'value2'] if used with split(;).
* `autosplit`: Automatically split delimited string into separate values; will split strings delimited by comma, semicolon, or space, e.g. 'value1,value2' => ['value1', 'value2'].
* `chop(x)`: Remove x characters off the end of value, e.g. chop(1): 'Value' => 'Valu'; when applied to a list, chops characters from each list value, e.g. chop(1): ['travel', 'beach']=> ['trave', 'beac'].
* `chomp(x)`: Remove x characters from the beginning of value, e.g. chomp(1): ['Value'] => ['alue']; when applied to a list, removes characters from each list value, e.g. chomp(1): ['travel', 'beach']=> ['ravel', 'each'].
* `sort`: Sort list of values, e.g. ['c', 'b', 'a'] => ['a', 'b', 'c'].
* `rsort`: Sort list of values in reverse order, e.g. ['a', 'b', 'c'] => ['c', 'b', 'a'].
* `reverse`: Reverse order of values, e.g. ['a', 'b', 'c'] => ['c', 'b', 'a'].
* `uniq`: Remove duplicate values, e.g. ['a', 'b', 'c', 'b', 'a'] => ['a', 'b', 'c'].
* `join(x)`: Join list of values with delimiter x, e.g. join(,): ['a', 'b', 'c'] => 'a,b,c'; the DELIM option functions similar to join(x) but with DELIM, the join happens before being passed to any filters.May optionally be used without an argument, that is 'join()' which joins values together with no delimiter. e.g. join(): ['a', 'b', 'c'] => 'abc'.
* `append(x)`: Append x to list of values, e.g. append(d): ['a', 'b', 'c'] => ['a', 'b', 'c', 'd'].
* `prepend(x)`: Prepend x to list of values, e.g. prepend(d): ['a', 'b', 'c'] => ['d', 'a', 'b', 'c'].
* `appends(x)`: Append s[tring] Append x to each value of list of values, e.g. appends(d): ['a', 'b', 'c'] => ['ad', 'bd', 'cd'].
* `prepends(x)`: Prepend s[tring] x to each value of list of values, e.g. prepends(d): ['a', 'b', 'c'] => ['da', 'db', 'dc'].
* `remove(x)`: Remove x from list of values, e.g. remove(b): ['a', 'b', 'c'] => ['a', 'c'].
* `slice(start:stop:step)`: Slice list using same semantics as Python's list slicing, e.g. slice(1:3): ['a', 'b', 'c', 'd'] => ['b', 'c']; slice(1:4:2): ['a', 'b', 'c', 'd'] => ['b', 'd']; slice(1:): ['a', 'b', 'c', 'd'] => ['b', 'c', 'd']; slice(:-1): ['a', 'b', 'c', 'd'] => ['a', 'b', 'c']; slice(::-1): ['a', 'b', 'c', 'd'] => ['d', 'c', 'b', 'a']. See also sslice().
* `sslice(start:stop:step)`: [s(tring) slice] Slice values in a list using same semantics as Python's string slicing, e.g. sslice(1:3):'abcd => 'bc'; sslice(1:4:2): 'abcd' => 'bd', etc. See also slice().
* `filter(x)`: Filter list of values using predicate x; for example, `{folder_album|filter(contains Events)}` returns only folders/albums containing the word 'Events' in their path.
* `int`: Convert values in list to integer, e.g. 1.0 => 1. If value cannot be converted to integer, remove value from list. ['1.1', 'x'] => ['1']. See also float.
* `float`: Convert values in list to floating point number, e.g. 1 => 1.0. If value cannot be converted to float, remove value from list. ['1', 'x'] => ['1.0']. See also int.
e.g. if Photo keywords are `["FOO","bar"]`:
- `"{keyword|lower}"` renders to `"foo", "bar"`
- `"{keyword|upper}"` renders to: `"FOO", "BAR"`
- `"{keyword|capitalize}"` renders to: `"Foo", "Bar"`
- `"{keyword|lower|parens}"` renders to: `"(foo)", "(bar)"`
* `"{keyword|lower}"` renders to `"foo", "bar"`
* `"{keyword|upper}"` renders to: `"FOO", "BAR"`
* `"{keyword|capitalize}"` renders to: `"Foo", "Bar"`
* `"{keyword|lower|parens}"` renders to: `"(foo)", "(bar)"`
e.g. if Photo description is "my description":
- `"{descr|titlecase}"` renders to: `"My Description"`
* `"{descr|titlecase}"` renders to: `"My Description"`
e.g. If Photo is in `Album1` in `Folder1`:
- `"{folder_album}"` renders to `["Folder1/Album1"]`
- `"{folder_album(>)}"` renders to `["Folder1>Album1"]`
- `"{folder_album()}"` renders to `["Folder1Album1"]`
* `"{folder_album}"` renders to `["Folder1/Album1"]`
* `"{folder_album(>)}"` renders to `["Folder1>Album1"]`
* `"{folder_album()}"` renders to `["Folder1Album1"]`
`[find,replace]`: optional text replacement to perform on rendered template value. For example, to replace "/" in an album name, you could use the template `"{album[/,-]}"`. Multiple replacements can be made by appending "|" and adding another find|replace pair. e.g. to replace both "/" and ":" in album name: `"{album[/,-|:,-]}"`. find/replace pairs are not limited to single characters. The "|" character cannot be used in a find/replace pair.
`conditional`: optional conditional expression that is evaluated as boolean (True/False) for use with the `?bool_value` modifier. Conditional expressions take the form '`not operator value`' where `not` is an optional modifier that negates the `operator`. Note: the space before the conditional expression is required if you use a conditional expression. Valid comparison operators are:
- `contains`: template field contains value, similar to python's `in`
- `matches`: template field contains exactly value, unlike `contains`: does not match partial matches
- `startswith`: template field starts with value
- `endswith`: template field ends with value
- `<=`: template field is less than or equal to value
- `>=`: template field is greater than or equal to value
- `<`: template field is less than value
- `>`: template field is greater than value
- `==`: template field equals value
- `!=`: template field does not equal value
* `contains`: template field contains value, similar to python's `in`
* `matches`: template field contains exactly value, unlike `contains`: does not match partial matches
* `startswith`: template field starts with value
* `endswith`: template field ends with value
* `<=`: template field is less than or equal to value
* `>=`: template field is greater than or equal to value
* `<`: template field is less than value
* `>`: template field is greater than value
* `==`: template field equals value
* `!=`: template field does not equal value
The `value` part of the conditional expression is treated as a bare (unquoted) word/phrase. Multiple values may be separated by '|' (the pipe symbol). `value` is itself a template statement so you can use one or more template fields in `value` which will be resolved before the comparison occurs.
For example:
- `{keyword matches Beach}` resolves to True if 'Beach' is a keyword. It would not match keyword 'BeachDay'.
- `{keyword contains Beach}` resolves to True if any keyword contains the word 'Beach' so it would match both 'Beach' and 'BeachDay'.
- `{photo.score.overall > 0.7}` resolves to True if the photo's overall aesthetic score is greater than 0.7.
- `{keyword|lower contains beach}` uses the lower case filter to do case-insensitive matching to match any keyword that contains the word 'beach'.
- `{keyword|lower not contains beach}` uses the `not` modifier to negate the comparison so this resolves to True if there is no keyword that matches 'beach'.
* `{keyword matches Beach}` resolves to True if 'Beach' is a keyword. It would not match keyword 'BeachDay'.
* `{keyword contains Beach}` resolves to True if any keyword contains the word 'Beach' so it would match both 'Beach' and 'BeachDay'.
* `{photo.score.overall > 0.7}` resolves to True if the photo's overall aesthetic score is greater than 0.7.
* `{keyword|lower contains beach}` uses the lower case filter to do case-insensitive matching to match any keyword that contains the word 'beach'.
* `{keyword|lower not contains beach}` uses the `not` modifier to negate the comparison so this resolves to True if there is no keyword that matches 'beach'.
Examples: to export photos that contain certain keywords with the `osxphotos export` command's `--directory` option:
@ -2236,24 +2252,24 @@ This renames any photo that is a favorite as 'Favorite-ImageName.jpg' (where 'Im
e.g. if photo is an HDR image,
- `"{hdr?ISHDR,NOTHDR}"` renders to `"ISHDR"`
* `"{hdr?ISHDR,NOTHDR}"` renders to `"ISHDR"`
and if it is not an HDR image,
- `"{hdr?ISHDR,NOTHDR}"` renders to `"NOTHDR"`
* `"{hdr?ISHDR,NOTHDR}"` renders to `"NOTHDR"`
`,default`: optional default value to use if the template name has no value. This modifier is also used for the value if False for boolean-type fields (see above) as well as to hold a sub-template for values like `{created.strftime}`. If no default value provided, "_" is used.
e.g., if photo has no title set,
- `"{title}"` renders to "_"
- `"{title,I have no title}"` renders to `"I have no title"`
* `"{title}"` renders to "_"
* `"{title,I have no title}"` renders to `"I have no title"`
Template fields such as `created.strftime` use the default value to pass the template to use for `strftime`.
e.g., if photo date is 4 February 2020, 19:07:38,
- `"{created.strftime,%Y-%m-%d-%H%M%S}"` renders to `"2020-02-04-190738"`
* `"{created.strftime,%Y-%m-%d-%H%M%S}"` renders to `"2020-02-04-190738"`
Some template fields such as `"{media_type}"` use the default value to allow customization of the output. For example, `"{media_type}"` resolves to the special media type of the photo such as `panorama` or `selfie`. You may use the default value to override these in form: `"{media_type,video=vidéo;time_lapse=vidéo_accélérée}"`. In this example, if photo was a time_lapse photo, `media_type` would resolve to `vidéo_accélérée` instead of `time_lapse`.
@ -2305,7 +2321,7 @@ cog.out(get_template_field_table())
|{created.hour}|2-digit hour of the photo creation time|
|{created.min}|2-digit minute of the photo creation time|
|{created.sec}|2-digit second of the photo creation time|
|{created.strftime}|Apply strftime template to file creation date/time. Should be used in form {created.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. {created.strftime,%Y-%U} would result in year-week number of year: '2020-23'. If used with no template will return null value. See https://strftime.org/ for help on strftime templates.|
|{created.strftime}|Apply strftime template to file creation date/time. Should be used in form {created.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. {created.strftime,%Y-%U} would result in year-week number of year: '2020-23'. If used with no template will return null value. See <https://strftime.org/> for help on strftime templates.|
|{modified}|Photo's modification date in ISO format, e.g. '2020-03-22'; uses creation date if photo is not modified|
|{modified.date}|Photo's modification date in ISO format, e.g. '2020-03-22'; uses creation date if photo is not modified|
|{modified.year}|4-digit year of photo modification time; uses creation date if photo is not modified|
@ -2319,7 +2335,7 @@ cog.out(get_template_field_table())
|{modified.hour}|2-digit hour of the photo modification time; uses creation date if photo is not modified|
|{modified.min}|2-digit minute of the photo modification time; uses creation date if photo is not modified|
|{modified.sec}|2-digit second of the photo modification time; uses creation date if photo is not modified|
|{modified.strftime}|Apply strftime template to file modification date/time. Should be used in form {modified.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. {modified.strftime,%Y-%U} would result in year-week number of year: '2020-23'. If used with no template will return null value. Uses creation date if photo is not modified. See https://strftime.org/ for help on strftime templates.|
|{modified.strftime}|Apply strftime template to file modification date/time. Should be used in form {modified.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. {modified.strftime,%Y-%U} would result in year-week number of year: '2020-23'. If used with no template will return null value. Uses creation date if photo is not modified. See <https://strftime.org/> for help on strftime templates.|
|{today}|Current date in iso format, e.g. '2020-03-22'|
|{today.date}|Current date in iso format, e.g. '2020-03-22'|
|{today.year}|4-digit year of current date|
@ -2333,7 +2349,7 @@ cog.out(get_template_field_table())
|{today.hour}|2-digit hour of the current date|
|{today.min}|2-digit minute of the current date|
|{today.sec}|2-digit second of the current date|
|{today.strftime}|Apply strftime template to current date/time. Should be used in form {today.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. {today.strftime,%Y-%U} would result in year-week number of year: '2020-23'. If used with no template will return null value. See https://strftime.org/ for help on strftime templates.|
|{today.strftime}|Apply strftime template to current date/time. Should be used in form {today.strftime,TEMPLATE} where TEMPLATE is a valid strftime template, e.g. {today.strftime,%Y-%U} would result in year-week number of year: '2020-23'. If used with no template will return null value. See <https://strftime.org/> for help on strftime templates.|
|{place.name}|Place name from the photo's reverse geolocation data, as displayed in Photos|
|{place.country_code}|The ISO country code from the photo's reverse geolocation data|
|{place.name.country}|Country name from the photo's reverse geolocation data|
@ -2385,17 +2401,17 @@ cog.out(get_template_field_table())
|{label}|Image categorization label associated with a photo (Photos 5+ only). Labels are added automatically by Photos using machine learning algorithms to categorize images. These are not the same as {keyword} which refers to the user-defined keywords/tags applied in Photos.|
|{label_normalized}|All lower case version of 'label' (Photos 5+ only)|
|{comment}|Comment(s) on shared Photos; format is 'Person name: comment text' (Photos 5+ only)|
|{exiftool}|Format: '{exiftool:GROUP:TAGNAME}'; use exiftool (https://exiftool.org) to extract metadata, in form GROUP:TAGNAME, from image. E.g. '{exiftool:EXIF:Make}' to get camera make, or {exiftool:IPTC:Keywords} to extract keywords. See https://exiftool.org/TagNames/ for list of valid tag names. You must specify group (e.g. EXIF, IPTC, etc) as used in `exiftool -G`. exiftool must be installed in the path to use this template.|
|{exiftool}|Format: '{exiftool:GROUP:TAGNAME}'; use exiftool (<https://exiftool.org>) to extract metadata, in form GROUP:TAGNAME, from image. E.g. '{exiftool:EXIF:Make}' to get camera make, or {exiftool:IPTC:Keywords} to extract keywords. See <https://exiftool.org/TagNames/> for list of valid tag names. You must specify group (e.g. EXIF, IPTC, etc) as used in `exiftool -G`. exiftool must be installed in the path to use this template.|
|{searchinfo.holiday}|Holiday names associated with a photo, e.g. 'Christmas Day'; (Photos 5+ only, applied automatically by Photos' image categorization algorithms).|
|{searchinfo.activity}|Activities associated with a photo, e.g. 'Sporting Event'; (Photos 5+ only, applied automatically by Photos' image categorization algorithms).|
|{searchinfo.venue}|Venues associated with a photo, e.g. name of restaurant; (Photos 5+ only, applied automatically by Photos' image categorization algorithms).|
|{searchinfo.venue_type}|Venue types associated with a photo, e.g. 'Restaurant'; (Photos 5+ only, applied automatically by Photos' image categorization algorithms).|
|{photo}|Provides direct access to the PhotoInfo object for the photo. Must be used in format '{photo.property}' where 'property' represents a PhotoInfo property. For example: '{photo.favorite}' is the same as '{favorite}' and '{photo.place.name}' is the same as '{place.name}'. '{photo}' provides access to properties that are not available as separate template fields but it assumes some knowledge of the underlying PhotoInfo class. See https://rhettbull.github.io/osxphotos/ for additional documentation on the PhotoInfo class.|
|{photo}|Provides direct access to the PhotoInfo object for the photo. Must be used in format '{photo.property}' where 'property' represents a PhotoInfo property. For example: '{photo.favorite}' is the same as '{favorite}' and '{photo.place.name}' is the same as '{place.name}'. '{photo}' provides access to properties that are not available as separate template fields but it assumes some knowledge of the underlying PhotoInfo class. See <https://rhettbull.github.io/osxphotos/> for additional documentation on the PhotoInfo class.|
|{detected_text}|List of text strings found in the image after performing text detection. Using '{detected_text}' will cause osxphotos to perform text detection on your photos using the built-in macOS text detection algorithms which will slow down your export. The results for each photo will be cached in the export database so that future exports with '--update' do not need to reprocess each photo. You may pass a confidence threshold value between 0.0 and 1.0 after a colon as in '{detected_text:0.5}'; The default confidence threshold is 0.75. '{detected_text}' works only on macOS Catalina (10.15) or later. Note: this feature is not the same thing as Live Text in macOS Monterey, which osxphotos does not yet support.|
|{shell_quote}|Use in form '{shell_quote,TEMPLATE}'; quotes the rendered TEMPLATE value(s) for safe usage in the shell, e.g. My file.jpeg => 'My file.jpeg'; only adds quotes if needed.|
|{strip}|Use in form '{strip,TEMPLATE}'; strips whitespace from begining and end of rendered TEMPLATE value(s).|
|{format}|Use in form, '{format:TYPE:FORMAT,TEMPLATE}'; converts TEMPLATE value to TYPE then formats the value using Python string formatting codes specified by FORMAT; TYPE is one of: 'int', 'float', or 'str'. For example, '{format:float:.1f,{exiftool:EXIF:FocalLength}}' will format focal length to 1 decimal place (e.g. '100.0'). |
|{function}|Execute a python function from an external file and use return value as template substitution. Use in format: {function:file.py::function_name} where 'file.py' is the name of the python file and 'function_name' is the name of the function to call. The function will be passed the PhotoInfo object for the photo. See https://github.com/RhetTbull/osxphotos/blob/master/examples/template_function.py for an example of how to implement a template function.|
|{function}|Execute a python function from an external file and use return value as template substitution. Use in format: {function:file.py::function_name} where 'file.py' is the name of the python file and 'function_name' is the name of the function to call. The function will be passed the PhotoInfo object for the photo. See <https://github.com/RhetTbull/osxphotos/blob/master/examples/template_function.py> for an example of how to implement a template function.|
<!--[[[end]]] -->
### <a name="exiftoolExifTool">ExifTool</a>

13
osxphotos/_constants.py
View File

@ -2,10 +2,14 @@
from __future__ import annotations
import logging
import os.path
import sqlite3
from datetime import datetime
from enum import Enum
logger: logging.Logger = logging.getLogger("osxphotos")
APP_NAME = "osxphotos"
OSXPHOTOS_URL = "https://github.com/RhetTbull/osxphotos"
@ -464,3 +468,12 @@ PROFILE_SORT_KEYS = [
UUID_PATTERN = (
r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
)
# Reference: https://docs.python.org/3/library/sqlite3.html?highlight=sqlite3%20threadsafety#sqlite3.threadsafety
# and https://docs.python.org/3/library/sqlite3.html?highlight=sqlite3%20threadsafety#sqlite3.connect
# 3: serialized mode; Threads may share the module, connections and cursors
# 3 is the default in the python.org python 3.11 distribution
# earlier versions of python.org python 3.x default to 1 which means threads may not share
# sqlite3 connections and thus PhotoInfo.export() cannot be used in a multithreaded environment
# pass SQLITE_CHECK_SAME_THREAD to sqlite3.connect() to enable multithreaded access on systems that support it
SQLITE_CHECK_SAME_THREAD = not sqlite3.threadsafety == 3
logger.debug(f"{SQLITE_CHECK_SAME_THREAD=}, {sqlite3.threadsafety=}")

117
osxphotos/albuminfo.py
View File

@ -18,6 +18,7 @@ from ._constants import (
_PHOTOS_4_VERSION,
_PHOTOS_5_ALBUM_KIND,
_PHOTOS_5_FOLDER_KIND,
_PHOTOS_5_VERSION,
TIME_DELTA,
AlbumSortOrder,
)
@ -61,7 +62,7 @@ class AlbumInfoBaseClass:
including folders, photos, etc.
"""
def __init__(self, db=None, uuid=None):
def __init__(self, db, uuid):
self._uuid = uuid
self._db = db
self._title = self._db._dbalbum_details[uuid]["title"]
@ -121,7 +122,8 @@ class AlbumInfoBaseClass:
@property
def end_date(self):
"""For Albums, return end date (most recent image) of album or None for albums with no images
For Import Sessions, return end date of import sessions (when import was completed)"""
For Import Sessions, return end date of import sessions (when import was completed)
"""
try:
return self._end_date
except AttributeError:
@ -163,6 +165,17 @@ class AlbumInfoBaseClass:
self._owner = None
return self._owner
def asdict(self):
"""Return album info as a dict"""
return {
"uuid": self.uuid,
"creation_date": self.creation_date,
"start_date": self.start_date,
"end_date": self.end_date,
"owner": self.owner,
"photos": [p.uuid for p in self.photos],
}
def __len__(self):
"""return number of photos contained in album"""
return len(self.photos)
@ -174,6 +187,10 @@ class AlbumInfo(AlbumInfoBaseClass):
including folders, photos, etc.
"""
def __init__(self, db, uuid):
super().__init__(db=db, uuid=uuid)
self._title = self._db._dbalbum_details[uuid]["title"]
@property
def title(self):
"""return title / name of album"""
@ -205,10 +222,11 @@ class AlbumInfo(AlbumInfoBaseClass):
@property
def folder_names(self):
"""return hierarchical list of folders the album is contained in
"""Return hierarchical list of folders the album is contained in
the folder list is in form:
["Top level folder", "sub folder 1", "sub folder 2", ...]
returns empty list if album is not in any folders"""
or empty list if album is not in any folders
"""
try:
return self._folder_names
@ -218,10 +236,9 @@ class AlbumInfo(AlbumInfoBaseClass):
@property
def folder_list(self):
"""return hierarchical list of folders the album is contained in
as list of FolderInfo objects in form
["Top level folder", "sub folder 1", "sub folder 2", ...]
returns empty list if album is not in any folders"""
"""Returns list of FolderInfo objects for each folder the album is contained in
or empty list if album is not in any folders
"""
try:
return self._folders
@ -246,7 +263,7 @@ class AlbumInfo(AlbumInfoBaseClass):
parent_pk = self._db._dbalbum_details[self._uuid]["parentfolder"]
self._parent = (
FolderInfo(db=self._db, uuid=self._db._dbalbums_pk[parent_pk])
if parent_pk != self._db._folder_root_pk
if parent_pk is not None and parent_pk != self._db._folder_root_pk
else None
)
return self._parent
@ -281,27 +298,80 @@ class AlbumInfo(AlbumInfoBaseClass):
f"Photo with uuid {photo.uuid} does not appear to be in this album"
)
def asdict(self):
"""Return album info as a dict"""
dict_data = super().asdict()
dict_data["title"] = self.title
dict_data["folder_names"] = self.folder_names
dict_data["folder_list"] = [f.uuid for f in self.folder_list]
dict_data["sort_order"] = self.sort_order
dict_data["parent"] = self.parent.uuid if self.parent else None
return dict_data
class ImportInfo(AlbumInfoBaseClass):
"""Information about import sessions"""
def __init__(self, db, uuid):
self._uuid = uuid
self._db = db
if self._db._db_version >= _PHOTOS_5_VERSION:
return super().__init__(db=db, uuid=uuid)
import_session = self._db._db_import_group[self._uuid]
try:
self._creation_date_timestamp = import_session[3]
except (ValueError, TypeError, KeyError):
self._creation_date_timestamp = datetime(1970, 1, 1)
self._start_date_timestamp = self._creation_date_timestamp
self._end_date_timestamp = self._creation_date_timestamp
self._title = import_session[2]
self._local_tz = get_local_tz(
datetime.fromtimestamp(self._creation_date_timestamp + TIME_DELTA)
)
@property
def title(self):
"""return title / name of import session"""
return self._title
@property
def photos(self):
"""return list of photos contained in import session"""
try:
return self._photos
except AttributeError:
uuid_list, sort_order = zip(
*[
(uuid, self._db._dbphotos[uuid]["fok_import_session"])
for uuid in self._db._dbphotos
if self._db._dbphotos[uuid]["import_uuid"] == self.uuid
if self._db._db_version >= _PHOTOS_5_VERSION:
uuid_list, sort_order = zip(
*[
(uuid, self._db._dbphotos[uuid]["fok_import_session"])
for uuid in self._db._dbphotos
if self._db._dbphotos[uuid]["import_uuid"] == self.uuid
]
)
sorted_uuid = sort_list_by_keys(uuid_list, sort_order)
self._photos = self._db.photos_by_uuid(sorted_uuid)
else:
import_photo_uuids = [
u
for u in self._db._dbphotos
if self._db._dbphotos[u]["import_uuid"] == self.uuid
]
)
sorted_uuid = sort_list_by_keys(uuid_list, sort_order)
self._photos = self._db.photos_by_uuid(sorted_uuid)
self._photos = self._db.photos_by_uuid(import_photo_uuids)
return self._photos
def asdict(self):
"""Return import info as a dict"""
return {
"uuid": self.uuid,
"creation_date": self.creation_date,
"start_date": self.start_date,
"end_date": self.end_date,
"title": self.title,
"photos": [p.uuid for p in self.photos],
}
def __bool__(self):
"""Always returns True
A photo without an import session will return None for import_info,
@ -309,6 +379,7 @@ class ImportInfo(AlbumInfoBaseClass):
"""
return True
class ProjectInfo(AlbumInfo):
"""
ProjectInfo with info about projects
@ -386,7 +457,7 @@ class FolderInfo:
parent_pk = self._db._dbalbum_details[self._uuid]["parentfolder"]
self._parent = (
FolderInfo(db=self._db, uuid=self._db._dbalbums_pk[parent_pk])
if parent_pk != self._db._folder_root_pk
if parent_pk is not None and parent_pk != self._db._folder_root_pk
else None
)
return self._parent
@ -416,6 +487,16 @@ class FolderInfo:
self._folders = folders
return self._folders
def asdict(self):
"""Return folder info as a dict"""
return {
"title": self.title,
"uuid": self.uuid,
"parent": self.parent.uuid if self.parent is not None else None,
"subfolders": [f.uuid for f in self.subfolders],
"albums": [a.uuid for a in self.album_info],
}
def __len__(self):
"""returns count of folders + albums contained in the folder"""
return len(self.subfolders) + len(self.album_info)

7
osxphotos/cli/import_cli.py
View File

@ -25,7 +25,7 @@ from rich.console import Console
from rich.markdown import Markdown
from strpdatetime import strpdatetime
from osxphotos._constants import _OSXPHOTOS_NONE_SENTINEL
from osxphotos._constants import _OSXPHOTOS_NONE_SENTINEL, SQLITE_CHECK_SAME_THREAD
from osxphotos._version import __version__
from osxphotos.cli.cli_params import TIMESTAMP_OPTION, VERBOSE_OPTION
from osxphotos.cli.common import get_data_dir
@ -77,7 +77,8 @@ def echo(message, emoji=True, **kwargs):
class PhotoInfoFromFile:
"""Mock PhotoInfo class for a file to be imported
Returns None for most attributes but allows some templates like exiftool and created to work correctly"""
Returns None for most attributes but allows some templates like exiftool and created to work correctly
"""
def __init__(self, filepath: Union[str, Path], exiftool: Optional[str] = None):
self._path = str(filepath)
@ -745,7 +746,7 @@ def write_sqlite_report(
file_exists = os.path.isfile(report_file)
conn = sqlite3.connect(report_file)
conn = sqlite3.connect(report_file, check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
if not append or not file_exists:

5
osxphotos/cli/report_writer.py
View File

@ -12,6 +12,7 @@ from abc import ABC, abstractmethod
from contextlib import suppress
from typing import Dict, Union
from osxphotos._constants import SQLITE_CHECK_SAME_THREAD
from osxphotos.export_db import OSXPHOTOS_ABOUT_STRING
from osxphotos.photoexporter import ExportResults
from osxphotos.sqlite_utils import sqlite_columns
@ -181,7 +182,7 @@ class ExportReportWriterSQLite(ReportWriterABC):
with suppress(FileNotFoundError):
os.unlink(self.output_file)
self._conn = sqlite3.connect(self.output_file)
self._conn = sqlite3.connect(self.output_file, check_same_thread=SQLITE_CHECK_SAME_THREAD)
self._create_tables()
self.report_id = self._generate_report_id()
@ -533,7 +534,7 @@ class SyncReportWriterSQLite(ReportWriterABC):
with suppress(FileNotFoundError):
os.unlink(self.output_file)
self._conn = sqlite3.connect(self.output_file)
self._conn = sqlite3.connect(self.output_file, check_same_thread=SQLITE_CHECK_SAME_THREAD)
self._create_tables()
self.report_id = self._generate_report_id()

1170
osxphotos/export_db.py
View File

File diff suppressed because it is too large Load Diff

24
osxphotos/export_db_utils.py
View File

@ -16,7 +16,7 @@ from rich import print
from osxphotos.photoinfo import PhotoInfo
from ._constants import OSXPHOTOS_EXPORT_DB
from ._constants import OSXPHOTOS_EXPORT_DB, SQLITE_CHECK_SAME_THREAD
from ._version import __version__
from .configoptions import ConfigOptions
from .export_db import OSXPHOTOS_EXPORTDB_VERSION, ExportDB
@ -50,7 +50,7 @@ def export_db_get_version(
dbfile: Union[str, pathlib.Path]
) -> Tuple[Optional[int], Optional[int]]:
"""returns version from export database as tuple of (osxphotos version, export_db version)"""
conn = sqlite3.connect(str(dbfile))
conn = sqlite3.connect(str(dbfile), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
if row := c.execute(
"SELECT osxphotos, exportdb FROM version ORDER BY id DESC LIMIT 1;"
@ -61,7 +61,7 @@ def export_db_get_version(
def export_db_vacuum(dbfile: Union[str, pathlib.Path]) -> None:
"""Vacuum export database"""
conn = sqlite3.connect(str(dbfile))
conn = sqlite3.connect(str(dbfile), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
c.execute("VACUUM;")
conn.commit()
@ -79,7 +79,7 @@ def export_db_update_signatures(
"""
export_dir = pathlib.Path(export_dir)
fileutil = FileUtil
conn = sqlite3.connect(str(dbfile))
conn = sqlite3.connect(str(dbfile), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
c.execute("SELECT filepath_normalized, filepath FROM export_data;")
rows = c.fetchall()
@ -114,7 +114,7 @@ def export_db_get_last_run(
export_db: Union[str, pathlib.Path]
) -> Tuple[Optional[str], Optional[str]]:
"""Get last run from export database"""
conn = sqlite3.connect(str(export_db))
conn = sqlite3.connect(str(export_db), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
if row := c.execute(
"SELECT datetime, args FROM runs ORDER BY id DESC LIMIT 1;"
@ -127,7 +127,7 @@ def export_db_get_errors(
export_db: Union[str, pathlib.Path]
) -> Tuple[Optional[str], Optional[str]]:
"""Get errors from export database"""
conn = sqlite3.connect(str(export_db))
conn = sqlite3.connect(str(export_db), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
results = c.execute(
"SELECT filepath, uuid, timestamp, error FROM export_data WHERE error is not null ORDER BY timestamp DESC;"
@ -145,7 +145,7 @@ def export_db_save_config_to_file(
"""Save export_db last run config to file"""
export_db = pathlib.Path(export_db)
config_file = pathlib.Path(config_file)
conn = sqlite3.connect(str(export_db))
conn = sqlite3.connect(str(export_db), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
row = c.execute("SELECT config FROM config ORDER BY id DESC LIMIT 1;").fetchone()
if not row:
@ -163,7 +163,7 @@ def export_db_get_config(
export_db: path to export database
override: if True, any loaded config values will overwrite existing values in config
"""
conn = sqlite3.connect(str(export_db))
conn = sqlite3.connect(str(export_db), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
row = c.execute("SELECT config FROM config ORDER BY id DESC LIMIT 1;").fetchone()
return (
@ -184,7 +184,7 @@ def export_db_check_signatures(
"""
export_dir = pathlib.Path(export_dir)
fileutil = FileUtil
conn = sqlite3.connect(str(dbfile))
conn = sqlite3.connect(str(dbfile), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
c.execute("SELECT filepath_normalized, filepath FROM export_data;")
rows = c.fetchall()
@ -236,7 +236,7 @@ def export_db_touch_files(
)
exportdb.close()
conn = sqlite3.connect(str(dbfile))
conn = sqlite3.connect(str(dbfile), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
if row := c.execute(
"SELECT config FROM config ORDER BY id DESC LIMIT 1;"
@ -318,7 +318,7 @@ def export_db_migrate_photos_library(
and update the UUIDs in the export database
"""
verbose(f"Loading data from export database {dbfile}")
conn = sqlite3.connect(str(dbfile))
conn = sqlite3.connect(str(dbfile), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
results = c.execute("SELECT uuid, photoinfo FROM photoinfo;").fetchall()
exportdb_uuids = {}
@ -495,7 +495,7 @@ def export_db_get_last_library(dbpath: Union[str, pathlib.Path]) -> str:
str: name of library used to export from or "" if not found
"""
dbpath = pathlib.Path(dbpath)
conn = sqlite3.connect(str(dbpath))
conn = sqlite3.connect(str(dbpath), check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
if results := c.execute(
"""

169
osxphotos/frozen_photoinfo.py Normal file
View File

@ -0,0 +1,169 @@
"""Freeze a PhotoInfo object to allow it to be used in concurrent.futures."""
from __future__ import annotations
import datetime
import json
import logging
import os
import re
from types import SimpleNamespace
from typing import Any
from osxmetadata import OSXMetaData
import osxphotos
from ._constants import TEXT_DETECTION_CONFIDENCE_THRESHOLD
from .exiftool import ExifToolCaching, get_exiftool_path
from .phototemplate import PhotoTemplate, RenderOptions
from .text_detection import detect_text
def frozen_photoinfo_factory(photo: "osxphotos.photoinfo.PhotoInfo") -> SimpleNamespace:
"""Return a frozen SimpleNamespace object for a PhotoInfo object"""
photo_json = photo.json()
def _object_hook(d: dict[Any, Any]):
if not d:
return d
# if d key matches a ISO 8601 datetime ('2023-03-24T06:46:57.690786', '2019-07-04T16:24:01-07:00', '2019-07-04T16:24:01+07:00'), convert to datetime
# fromisoformat will also handle dates with timezone offset in form +0700, etc.
for k, v in d.items():
if isinstance(v, str) and re.match(
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}[.]?\d*[+-]?\d{2}[:]?\d{2}?", v
):
d[k] = datetime.datetime.fromisoformat(v)
return SimpleNamespace(**d)
frozen = json.loads(photo_json, object_hook=lambda d: _object_hook(d))
# add on json() method to frozen object
def _json(*args):
return photo_json
frozen.json = _json
# add hexdigest property to frozen object
frozen.hexdigest = photo.hexdigest
# add on detected_text method to frozen object
frozen = _add_detected_text(frozen)
# add on exiftool property to frozen object
frozen = _add_exiftool(frozen, photo)
# add on render_template method to frozen object
frozen = _add_render_template(frozen)
# add on the _db property to frozen object
# frozen objects don't really have a _db class but some things expect it (e.g. _db._beta)
frozen._db = SimpleNamespace(_beta=photo._db._beta)
return frozen
def _add_detected_text(frozen: SimpleNamespace) -> SimpleNamespace:
"""Add detected_text method to frozen PhotoInfo object"""
def detected_text(confidence_threshold=TEXT_DETECTION_CONFIDENCE_THRESHOLD):
"""Detects text in photo and returns lists of results as (detected text, confidence)
confidence_threshold: float between 0.0 and 1.0. If text detection confidence is below this threshold,
text will not be returned. Default is TEXT_DETECTION_CONFIDENCE_THRESHOLD
If photo is edited, uses the edited photo, otherwise the original; falls back to the preview image if neither edited or original is available
Returns: list of (detected text, confidence) tuples
"""
try:
return frozen._detected_text_cache[confidence_threshold]
except (AttributeError, KeyError) as e:
if isinstance(e, AttributeError):
frozen._detected_text_cache = {}
try:
detected_text = frozen._detected_text()
except Exception as e:
logging.warning(f"Error detecting text in photo {frozen.uuid}: {e}")
detected_text = []
frozen._detected_text_cache[confidence_threshold] = [
(text, confidence)
for text, confidence in detected_text
if confidence >= confidence_threshold
]
return frozen._detected_text_cache[confidence_threshold]
def _detected_text():
"""detect text in photo, either from cached extended attribute or by attempting text detection"""
path = (
frozen.path_edited
if frozen.hasadjustments and frozen.path_edited
else frozen.path
)
path = path or frozen.path_derivatives[0] if frozen.path_derivatives else None
if not path:
return []
md = OSXMetaData(path)
try:
def decoder(val):
"""Decode value from JSON"""
return json.loads(val.decode("utf-8"))
detected_text = md.get_xattr(
"osxphotos.metadata:detected_text", decode=decoder
)
except KeyError:
detected_text = None
if detected_text is None:
orientation = frozen.orientation or None
detected_text = detect_text(path, orientation)
def encoder(obj):
"""Encode value as JSON"""
val = json.dumps(obj)
return val.encode("utf-8")
md.set_xattr(
"osxphotos.metadata:detected_text", detected_text, encode=encoder
)
return detected_text
frozen.detected_text = detected_text
frozen._detected_text = _detected_text
return frozen
def _add_exiftool(
frozen: SimpleNamespace, photo: "osxphotos.photoinfo.PhotoInfo"
) -> SimpleNamespace:
"""Add exiftool property to frozen PhotoInfo object"""
frozen._exiftool_path = photo._db._exiftool_path or None
return frozen
def _add_render_template(frozen: SimpleNamespace) -> SimpleNamespace:
"""Add render_template method to frozen PhotoInfo object"""
def render_template(template_str: str, options: RenderOptions | None = None):
"""Renders a template string for PhotoInfo instance using PhotoTemplate
Args:
template_str: a template string with fields to render
options: a RenderOptions instance
Returns:
([rendered_strings], [unmatched]): tuple of list of rendered strings and list of unmatched template values
"""
options = options or RenderOptions()
template = PhotoTemplate(frozen, exiftool_path=frozen._exiftool_path)
return template.render(template_str, options)
frozen.render_template = render_template
return frozen

6
osxphotos/photodates.py
View File

@ -11,7 +11,7 @@ import photoscript
from strpdatetime import strpdatetime
from tenacity import retry, stop_after_attempt, wait_exponential
from ._constants import _DB_TABLE_NAMES
from ._constants import _DB_TABLE_NAMES, SQLITE_CHECK_SAME_THREAD
from .datetime_utils import (
datetime_has_tz,
datetime_remove_tz,
@ -219,7 +219,7 @@ def _set_date_added(library_path: str, uuid: str, date_added: datetime.datetime)
asset_table = _DB_TABLE_NAMES[photos_version]["ASSET"]
timestamp = datetime_to_photos_timestamp(date_added)
conn = sqlite3.connect(db_path)
conn = sqlite3.connect(db_path, check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
c.execute(
f"UPDATE {asset_table} SET ZADDEDDATE=? WHERE ZUUID=?",
@ -268,7 +268,7 @@ def get_photo_date_added(
photos_version = get_photos_library_version(library_path)
db_path = str(pathlib.Path(library_path) / "database/Photos.sqlite")
asset_table = _DB_TABLE_NAMES[photos_version]["ASSET"]
conn = sqlite3.connect(db_path)
conn = sqlite3.connect(db_path, check_same_thread=SQLITE_CHECK_SAME_THREAD)
c = conn.cursor()
c.execute(
f"SELECT ZADDEDDATE FROM {asset_table} WHERE ZUUID=?",

46
osxphotos/photoexporter.py
View File

@ -12,6 +12,7 @@ from collections import namedtuple # pylint: disable=syntax-error
from dataclasses import asdict, dataclass
from datetime import datetime
from enum import Enum
from types import SimpleNamespace
import photoscript
from mako.template import Template
@ -31,7 +32,7 @@ from ._constants import (
)
from ._version import __version__
from .datetime_utils import datetime_tz_to_utc
from .exiftool import ExifTool, exiftool_can_write
from .exiftool import ExifTool, ExifToolCaching, exiftool_can_write, get_exiftool_path
from .export_db import ExportDB, ExportDBTemp
from .fileutil import FileUtil
from .photokit import (
@ -68,6 +69,7 @@ if t.TYPE_CHECKING:
# retry if download_missing/use_photos_export fails the first time (which sometimes it does)
MAX_PHOTOSCRIPT_RETRIES = 3
# return values for _should_update_photo
class ShouldUpdate(Enum):
NOT_IN_DATABASE = 1
@ -309,7 +311,6 @@ class ExportResults:
xattr_skipped=None,
xattr_written=None,
):
local_vars = locals()
self._datetime = datetime.now().isoformat()
for attr in self.attributes:
@ -374,7 +375,7 @@ class PhotoExporter:
def __init__(self, photo: "PhotoInfo", tmpdir: t.Optional[str] = None):
self.photo = photo
self._render_options = RenderOptions()
self._verbose = self.photo._verbose
self._verbose = photo._verbose
# define functions for adding markup
self._filepath = add_rich_markup_tag("filepath", rich=False)
@ -950,7 +951,8 @@ class PhotoExporter:
"""Stage a photo for export with AppleScript to a temporary directory
Note: If exporting an edited live photo, the associated live video will not be exported.
This is a limitation of the Photos AppleScript interface and Photos behaves the same way."""
This is a limitation of the Photos AppleScript interface and Photos behaves the same way.
"""
if options.edited and not self.photo.hasadjustments:
raise ValueError("Edited version requested but photo has no adjustments")
@ -1564,7 +1566,7 @@ class PhotoExporter:
with ExifTool(
filepath,
flags=options.exiftool_flags,
exiftool=self.photo._db._exiftool_path,
exiftool=self.photo._exiftool_path,
) as exiftool:
for exiftag, val in exif_info.items():
if type(val) == list:
@ -1744,7 +1746,6 @@ class PhotoExporter:
elif self.photo.ismovie:
exif["Keys:GPSCoordinates"] = f"{lat} {lon}"
exif["UserData:GPSCoordinates"] = f"{lat} {lon}"
# process date/time and timezone offset
# Photos exports the following fields and sets modify date to creation date
# [EXIF] Modify Date : 2020:10:30 00:00:00
@ -1854,7 +1855,7 @@ class PhotoExporter:
def _get_exif_keywords(self):
"""returns list of keywords found in the file's exif metadata"""
keywords = []
exif = self.photo.exiftool
exif = exiftool_caching(self.photo)
if exif:
exifdict = exif.asdict()
for field in ["IPTC:Keywords", "XMP:TagsList", "XMP:Subject"]:
@ -1871,7 +1872,7 @@ class PhotoExporter:
def _get_exif_persons(self):
"""returns list of persons found in the file's exif metadata"""
persons = []
exif = self.photo.exiftool
exif = exiftool_caching(self.photo)
if exif:
exifdict = exif.asdict()
try:
@ -2142,3 +2143,32 @@ def rename_jpeg_files(files, jpeg_ext, fileutil):
else:
new_files.append(file)
return new_files
def exiftool_caching(photo: SimpleNamespace) -> ExifToolCaching:
"""Return ExifToolCaching object for photo
Args:
photo: SimpleNamespace object with photo info
Returns:
ExifToolCaching object
"""
try:
return photo._exiftool_caching
except AttributeError:
try:
exiftool_path = photo._exiftool_path or get_exiftool_path()
if photo.path is not None and os.path.isfile(photo.path):
exiftool = ExifToolCaching(photo.path, exiftool=exiftool_path)
else:
exiftool = None
except FileNotFoundError:
# get_exiftool_path raises FileNotFoundError if exiftool not found
exiftool = None
logging.warning(
"exiftool not in path; download and install from https://exiftool.org/"
)
photo._exiftool_caching = exiftool
return photo._exiftool_caching

313
osxphotos/photoinfo.py
View File

@ -8,14 +8,16 @@ import contextlib
import dataclasses
import datetime
import json
import logging
import os
import os.path
import pathlib
import plistlib
import re
from datetime import timedelta, timezone
from functools import cached_property
from types import SimpleNamespace
from typing import Any, Dict, Optional
import logging
import yaml
from osxmetadata import OSXMetaData
@ -66,7 +68,7 @@ from .text_detection import detect_text
from .uti import get_preferred_uti_extension, get_uti_for_extension
from .utils import _get_resource_loc, hexdigest, list_directory
__all__ = ["PhotoInfo", "PhotoInfoNone"]
__all__ = ["PhotoInfo", "PhotoInfoNone", "frozen_photoinfo_factory"]
logger = logging.getLogger("osxphotos")
@ -81,6 +83,7 @@ class PhotoInfo:
self._uuid: str = uuid
self._info: dict[str, Any] = info
self._db: "osxphotos.PhotosDB" = db
self._exiftool_path = self._db._exiftool_path
self._verbose = self._db._verbose
@property
@ -388,6 +391,8 @@ class PhotoInfo:
"""return path_edited_live_photo for Photos <= 4"""
if self._db._db_version > _PHOTOS_4_VERSION:
raise RuntimeError("Wrong database format!")
if not self.live_photo:
return None
photopath = self._get_predicted_path_edited_live_photo_4()
if photopath is not None and not os.path.isfile(photopath):
# the heuristic failed, so try to find the file
@ -401,10 +406,6 @@ class PhotoInfo:
),
None,
)
if photopath is None:
logger.debug(
f"MISSING PATH: edited live photo file for UUID {self._uuid} does not appear to exist"
)
return photopath
def _path_edited_5_live_photo(self):
@ -1198,7 +1199,6 @@ class PhotoInfo:
"""
if self._db._db_version <= _PHOTOS_4_VERSION:
logger.debug(f"score not implemented for this database version")
return None
try:
@ -1344,7 +1344,6 @@ class PhotoInfo:
"""
if self._db._db_version <= _PHOTOS_4_VERSION:
logger.debug(f"exif_info not implemented for this database version")
return None
try:
@ -1427,11 +1426,14 @@ class PhotoInfo:
def hexdigest(self):
"""Returns a unique digest of the photo's properties and metadata;
useful for detecting changes in any property/metadata of the photo"""
return hexdigest(self.json())
return hexdigest(self._json_hexdigest())
@cached_property
def cloud_metadata(self) -> Dict:
"""Returns contents of ZCLOUDMASTERMEDIAMETADATA as dict"""
def cloud_metadata(self) -> dict[Any, Any]:
"""Returns contents of ZCLOUDMASTERMEDIAMETADATA as dict; Photos 5+ only"""
if self._db._db_version <= _PHOTOS_4_VERSION:
return {}
# This is a large blob of data so don't load it unless requested
asset_table = _DB_TABLE_NAMES[self._db._photos_ver]["ASSET"]
sql_cloud_metadata = f"""
@ -1442,10 +1444,6 @@ class PhotoInfo:
WHERE {asset_table}.ZUUID = ?
"""
if self._db._db_version <= _PHOTOS_4_VERSION:
logger.debug(f"cloud_metadata not implemented for this database version")
return {}
_, cursor = self._db.get_db_connection()
metadata = {}
if results := cursor.execute(sql_cloud_metadata, (self.uuid,)).fetchone():
@ -1782,80 +1780,112 @@ class PhotoInfo:
def asdict(self):
"""return dict representation"""
folders = {album.title: album.folder_names for album in self.album_info}
exif = dataclasses.asdict(self.exif_info) if self.exif_info else {}
place = self.place.asdict() if self.place else {}
score = dataclasses.asdict(self.score) if self.score else {}
adjustments = self.adjustments.asdict() if self.adjustments else {}
album_info = [album.asdict() for album in self.album_info]
burst_album_info = [a.asdict() for a in self.burst_album_info]
burst_photos = [p.uuid for p in self.burst_photos]
comments = [comment.asdict() for comment in self.comments]
exif_info = dataclasses.asdict(self.exif_info) if self.exif_info else {}
face_info = [face.asdict() for face in self.face_info]
folders = {album.title: album.folder_names for album in self.album_info}
import_info = self.import_info.asdict() if self.import_info else {}
likes = [like.asdict() for like in self.likes]
faces = [face.asdict() for face in self.face_info]
person_info = [p.asdict() for p in self.person_info]
place = self.place.asdict() if self.place else {}
project_info = [p.asdict() for p in self.project_info]
score = dataclasses.asdict(self.score) if self.score else {}
search_info = self.search_info.asdict() if self.search_info else {}
search_info_normalized = (
self.search_info_normalized.asdict() if self.search_info_normalized else {}
)
return {
"library": self._db._library_path,
"uuid": self.uuid,
"filename": self.filename,
"original_filename": self.original_filename,
"adjustments": adjustments,
"album_info": album_info,
"albums": self.albums,
"burst_album_info": burst_album_info,
"burst_albums": self.burst_albums,
"burst_default_pick": self.burst_default_pick,
"burst_key": self.burst_key,
"burst_photos": burst_photos,
"burst_selected": self.burst_selected,
"burst": self.burst,
"cloud_guid": self.cloud_guid,
"cloud_metadata": self.cloud_metadata,
"cloud_owner_hashed_id": self.cloud_owner_hashed_id,
"comments": comments,
"date_added": self.date_added,
"date_modified": self.date_modified,
"date_trashed": self.date_trashed,
"date": self.date,
"description": self.description,
"title": self.title,
"keywords": self.keywords,
"labels": self.labels,
"keywords": self.keywords,
"albums": self.albums,
"folders": folders,
"persons": self.persons,
"faces": faces,
"path": self.path,
"ismissing": self.ismissing,
"hasadjustments": self.hasadjustments,
"exif_info": exif_info,
"external_edit": self.external_edit,
"face_info": face_info,
"favorite": self.favorite,
"filename": self.filename,
"fingerprint": self.fingerprint,
"folders": folders,
"has_raw": self.has_raw,
"hasadjustments": self.hasadjustments,
"hdr": self.hdr,
"height": self.height,
"hidden": self.hidden,
"latitude": self._latitude,
"longitude": self._longitude,
"path_edited": self.path_edited,
"shared": self.shared,
"isphoto": self.isphoto,
"ismovie": self.ismovie,
"uti": self.uti,
"uti_original": self.uti_original,
"burst": self.burst,
"live_photo": self.live_photo,
"path_live_photo": self.path_live_photo,
"iscloudasset": self.iscloudasset,
"import_info": import_info,
"incloud": self.incloud,
"intrash": self.intrash,
"iscloudasset": self.iscloudasset,
"ismissing": self.ismissing,
"ismovie": self.ismovie,
"isphoto": self.isphoto,
"israw": self.israw,
"isreference": self.isreference,
"date_modified": self.date_modified,
"keywords": self.keywords,
"labels_normalized": self.labels_normalized,
"labels": self.labels,
"latitude": self._latitude,
"library": self._db._library_path,
"likes": likes,
"live_photo": self.live_photo,
"location": self.location,
"longitude": self._longitude,
"orientation": self.orientation,
"original_filename": self.original_filename,
"original_filesize": self.original_filesize,
"original_height": self.original_height,
"original_orientation": self.original_orientation,
"original_width": self.original_width,
"owner": self.owner,
"panorama": self.panorama,
"path_derivatives": self.path_derivatives,
"path_edited_live_photo": self.path_edited_live_photo,
"path_edited": self.path_edited,
"path_live_photo": self.path_live_photo,
"path_raw": self.path_raw,
"path": self.path,
"person_info": person_info,
"persons": self.persons,
"place": place,
"portrait": self.portrait,
"project_info": project_info,
"raw_original": self.raw_original,
"score": score,
"screenshot": self.screenshot,
"search_info_normalized": search_info_normalized,
"search_info": search_info,
"selfie": self.selfie,
"shared": self.shared,
"slow_mo": self.slow_mo,
"time_lapse": self.time_lapse,
"hdr": self.hdr,
"selfie": self.selfie,
"panorama": self.panorama,
"has_raw": self.has_raw,
"israw": self.israw,
"raw_original": self.raw_original,
"title": self.title,
"tzoffset": self.tzoffset,
"uti_edited": self.uti_edited,
"uti_original": self.uti_original,
"uti_raw": self.uti_raw,
"path_raw": self.path_raw,
"place": place,
"exif": exif,
"score": score,
"intrash": self.intrash,
"height": self.height,
"uti": self.uti,
"uuid": self.uuid,
"visible": self.visible,
"width": self.width,
"orientation": self.orientation,
"original_height": self.original_height,
"original_width": self.original_width,
"original_orientation": self.original_orientation,
"original_filesize": self.original_filesize,
"comments": comments,
"likes": likes,
"search_info": search_info,
"fingerprint": self.fingerprint,
"cloud_guid": self.cloud_guid,
"cloud_owner_hashed_id": self.cloud_owner_hashed_id,
}
def json(self):
@ -1867,8 +1897,44 @@ class PhotoInfo:
dict_data = self.asdict()
for k, v in dict_data.items():
# sort lists such as keywords so JSON is consistent
# but do not sort certain items like location
if k in ["location"]:
continue
if v and isinstance(v, (list, tuple)) and not isinstance(v[0], dict):
dict_data[k] = sorted(v)
dict_data[k] = sorted(v, key=lambda v: v if v is not None else "")
return json.dumps(dict_data, sort_keys=True, default=default)
def _json_hexdigest(self):
"""JSON for use by hexdigest()"""
# This differs from json() because hexdigest must not change if metadata changed
# With json(), sort order of lists of dicts is not consistent but these aren't needed
# for computing hexdigest so we can ignore them
# also don't use visible because it changes based on Photos UI state
def default(o):
if isinstance(o, (datetime.date, datetime.datetime)):
return o.isoformat()
dict_data = self.asdict()
for k in [
"album_info",
"burst_album_info",
"face_info",
"person_info",
"visible",
]:
del dict_data[k]
for k, v in dict_data.items():
# sort lists such as keywords so JSON is consistent
# but do not sort certain items like location
if k in ["location"]:
continue
if v and isinstance(v, (list, tuple)) and not isinstance(v[0], dict):
dict_data[k] = sorted(v, key=lambda v: v if v is not None else "")
return json.dumps(dict_data, sort_keys=True, default=default)
def __eq__(self, other):
@ -1893,10 +1959,111 @@ class PhotoInfo:
class PhotoInfoNone:
"""mock class that returns None for all attributes"""
"""Mock class that returns None for all attributes"""
def __init__(self):
pass
def __getattribute__(self, name):
return None
def frozen_photoinfo_factory(photo: PhotoInfo) -> SimpleNamespace:
"""Return a frozen SimpleNamespace object for a PhotoInfo object"""
photo_json = photo.json()
def _object_hook(d: dict[Any, Any]):
if not d:
return d
# if d key matches a ISO 8601 datetime ('2023-03-24T06:46:57.690786', '2019-07-04T16:24:01-07:00', '2019-07-04T16:24:01+07:00'), convert to datetime
# fromisoformat will also handle dates with timezone offset in form +0700, etc.
for k, v in d.items():
if isinstance(v, str) and re.match(
r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}[.]?\d*[+-]?\d{2}[:]?\d{2}?", v
):
d[k] = datetime.datetime.fromisoformat(v)
return SimpleNamespace(**d)
frozen = json.loads(photo_json, object_hook=lambda d: _object_hook(d))
# add on json() method to frozen object
def _json(*args):
return photo_json
frozen.json = _json
# add hexdigest property to frozen object
frozen.hexdigest = photo.hexdigest
def detected_text(confidence_threshold=TEXT_DETECTION_CONFIDENCE_THRESHOLD):
"""Detects text in photo and returns lists of results as (detected text, confidence)
confidence_threshold: float between 0.0 and 1.0. If text detection confidence is below this threshold,
text will not be returned. Default is TEXT_DETECTION_CONFIDENCE_THRESHOLD
If photo is edited, uses the edited photo, otherwise the original; falls back to the preview image if neither edited or original is available
Returns: list of (detected text, confidence) tuples
"""
try:
return frozen._detected_text_cache[confidence_threshold]
except (AttributeError, KeyError) as e:
if isinstance(e, AttributeError):
frozen._detected_text_cache = {}
try:
detected_text = frozen._detected_text()
except Exception as e:
logging.warning(f"Error detecting text in photo {frozen.uuid}: {e}")
detected_text = []
frozen._detected_text_cache[confidence_threshold] = [
(text, confidence)
for text, confidence in detected_text
if confidence >= confidence_threshold
]
return frozen._detected_text_cache[confidence_threshold]
def _detected_text():
"""detect text in photo, either from cached extended attribute or by attempting text detection"""
path = (
frozen.path_edited
if frozen.hasadjustments and frozen.path_edited
else frozen.path
)
path = path or frozen.path_derivatives[0] if frozen.path_derivatives else None
if not path:
return []
md = OSXMetaData(path)
try:
def decoder(val):
"""Decode value from JSON"""
return json.loads(val.decode("utf-8"))
detected_text = md.get_xattr(
"osxphotos.metadata:detected_text", decode=decoder
)
except KeyError:
detected_text = None
if detected_text is None:
orientation = frozen.orientation or None
detected_text = detect_text(path, orientation)
def encoder(obj):
"""Encode value as JSON"""
val = json.dumps(obj)
return val.encode("utf-8")
md.set_xattr(
"osxphotos.metadata:detected_text", detected_text, encode=encoder
)
return detected_text
frozen.detected_text = detected_text
frozen._detected_text = _detected_text
return frozen

3
osxphotos/photosdb/photosdb.py
View File

@ -284,6 +284,9 @@ class PhotosDB:
# key is Z_PK of ZMOMENT table and values are the moment info
self._db_moment_pk = {}
# Dict to hold data on imports for Photos <= 4
self._db_import_group = {}
logger.debug(f"dbfile = {dbfile}")
if dbfile is None:

2
osxphotos/phototemplate.py
View File

@ -1352,7 +1352,7 @@ class PhotoTemplate:
subfield = subfield.lower()
if subfield in exifdict:
values = exifdict[subfield]
values = [values] if not isinstance(values, list) else values
values = values if isinstance(values, list) else [values]
values = [str(v) for v in values]
# sanitize directory names if needed

14
osxphotos/phototz.py
View File

@ -11,7 +11,7 @@ from typing import Callable, Optional, Tuple
from photoscript import Photo
from tenacity import retry, stop_after_attempt, wait_exponential
from ._constants import _DB_TABLE_NAMES
from ._constants import _DB_TABLE_NAMES, SQLITE_CHECK_SAME_THREAD
from .photosdb.photosdb_utils import get_photos_library_version
from .timezones import Timezone
from .utils import get_last_library_path, get_system_library_path, noop
@ -67,7 +67,9 @@ class PhotoTimeZone:
ON ZADDITIONALASSETATTRIBUTES.ZASSET = {self.ASSET_TABLE}.Z_PK
WHERE {self.ASSET_TABLE}.ZUUID = '{uuid}'
"""
with sqlite3.connect(self.db_path) as conn:
with sqlite3.connect(
self.db_path, check_same_thread=SQLITE_CHECK_SAME_THREAD
) as conn:
c = conn.cursor()
c.execute(sql)
results = c.fetchone()
@ -137,7 +139,9 @@ class PhotoTimeZoneUpdater:
ON ZADDITIONALASSETATTRIBUTES.ZASSET = {self.ASSET_TABLE}.Z_PK
WHERE {self.ASSET_TABLE}.ZUUID = '{uuid}'
"""
with sqlite3.connect(self.db_path) as conn:
with sqlite3.connect(
self.db_path, check_same_thread=SQLITE_CHECK_SAME_THREAD
) as conn:
c = conn.cursor()
c.execute(sql)
results = c.fetchone()
@ -151,7 +155,9 @@ class PhotoTimeZoneUpdater:
ZTIMEZONENAME='{self.tz_name}'
WHERE Z_PK={z_pk};
"""
with sqlite3.connect(self.db_path) as conn:
with sqlite3.connect(
self.db_path, check_same_thread=SQLITE_CHECK_SAME_THREAD
) as conn:
c = conn.cursor()
c.execute(sql_update)
conn.commit()

10
osxphotos/sqlite_utils.py
View File

@ -5,14 +5,22 @@ import pathlib
import sqlite3
from typing import List, Tuple
from ._constants import SQLITE_CHECK_SAME_THREAD
logger = logging.getLogger("osxphotos")
def sqlite_open_ro(dbname: str) -> Tuple[sqlite3.Connection, sqlite3.Cursor]:
"""opens sqlite file dbname in read-only mode
returns tuple of (connection, cursor)"""
try:
dbpath = pathlib.Path(dbname).resolve()
conn = sqlite3.connect(f"{dbpath.as_uri()}?mode=ro", timeout=1, uri=True)
conn = sqlite3.connect(
f"{dbpath.as_uri()}?mode=ro",
timeout=1,
uri=True,
check_same_thread=SQLITE_CHECK_SAME_THREAD,
)
c = conn.cursor()
except sqlite3.Error as e:
raise sqlite3.Error(

6
osxphotos/sqlitekvstore.py
View File

@ -9,6 +9,8 @@ from typing import Callable, Dict, Generator, Iterable, Optional, Tuple, TypeVar
# keep mypy happy, keys/values can be any type supported by SQLite
T = TypeVar("T")
__version__ = "0.3.0"
__all__ = ["SQLiteKVStore"]
@ -41,7 +43,7 @@ class SQLiteKVStore:
self._serialize_func = serialize
self._deserialize_func = deserialize
self._conn = (
sqlite3.Connection(dbpath)
sqlite3.connect(dbpath)
if os.path.exists(dbpath)
else self._create_database(dbpath)
)
@ -53,7 +55,7 @@ class SQLiteKVStore:
def _create_database(self, dbpath: str):
"""Create the key-value database"""
conn = sqlite3.Connection(dbpath)
conn = sqlite3.connect(dbpath)
cursor = conn.cursor()
cursor.execute(
"""CREATE TABLE IF NOT EXISTS _about (

2
tests/test_cli.py
View File

@ -4631,7 +4631,7 @@ def test_export_force_update():
conn = sqlite3.connect(dbpath)
c = conn.cursor()
except sqlite3.Error as e:
pytest.exit(f"An error occurred opening sqlite file")
pytest.exit("An error occurred opening sqlite file")
# photo is IMG_4547.jpg
c.execute(

66
tests/test_concurrent_export.py Normal file
View File

@ -0,0 +1,66 @@
""""Test that PhotoInfo.export can export concurrently"""
import concurrent.futures
import pathlib
import sqlite3
import tempfile
import pytest
import osxphotos
PHOTOS_DB = "tests/Test-10.15.7.photoslibrary"
@pytest.mark.skipif(sqlite3.threadsafety != 3, reason="sqlite3 not threadsafe")
@pytest.mark.parametrize(
"count", range(10)
) # repeat multiple times to try to catch any concurrency errors
def test_concurrent_export(count):
"""Test that PhotoInfo.export can export concurrently"""
photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB)
photos = [p for p in photosdb.photos() if not p.ismissing]
with tempfile.TemporaryDirectory() as tmpdir:
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:
futures = [
executor.submit(p.export, tmpdir, f"{p.uuid}_{p.original_filename}")
for p in photos
]
exported = []
for future in concurrent.futures.as_completed(futures):
exported.extend(future.result())
assert len(exported) == len(photos)
@pytest.mark.skipif(sqlite3.threadsafety != 3, reason="sqlite3 not threadsafe")
@pytest.mark.parametrize(
"count", range(10)
) # repeat multiple times to try to catch any concurrency errors
def test_concurrent_export_with_exportdb(count):
"""Test that PhotoInfo.export can export concurrently"""
photosdb = osxphotos.PhotosDB(dbfile=PHOTOS_DB)
photos = [p for p in photosdb.photos() if not p.ismissing]
with tempfile.TemporaryDirectory() as tmpdir:
exportdb = osxphotos.ExportDB(pathlib.Path(tmpdir) / "export.db", tmpdir)
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:
futures = []
for p in photos:
options = osxphotos.ExportOptions()
options.export_db = exportdb
exporter = osxphotos.PhotoExporter(p)
futures.append(
executor.submit(
exporter.export,
tmpdir,
f"{p.uuid}_{p.original_filename}",
options=options,
)
)
export_results = osxphotos.photoexporter.ExportResults()
for future in concurrent.futures.as_completed(futures):
export_results += future.result()
assert len(export_results.exported) == len(photos)
assert len(list(exportdb.get_exported_files())) == len(photos)