Compare commits

..

9 Commits

Author SHA1 Message Date
Rhet Turnbull
965e10e20f Fixed test for M1, added about command, closes #315 2021-01-12 07:03:07 -08:00
Rhet Turnbull
61f649e59d Update @narensankar0529 as a contributor 2021-01-12 06:19:40 -08:00
Rhet Turnbull
165f9b08f5 Fixed time zone for tests 2021-01-11 21:42:03 -08:00
Rhet Turnbull
039118c1aa Add @narensankar0529 as a contributor 2021-01-11 21:33:06 -08:00
Rhet Turnbull
27f779b16c Added version check for M1 macs 2021-01-11 20:27:51 -08:00
Rhet Turnbull
eec960861e Updated CHANGELOG.md, [skip ci] 2021-01-11 06:49:26 -08:00
Rhet Turnbull
4d924d0826 Completed implementation of --jpeg-ext, fixed --dry-run, closes #330, #346 2021-01-11 06:45:35 -08:00
Rhet Turnbull
55c088eea2 Added --jpeg-ext, implements #330 2021-01-10 09:44:42 -08:00
Rhet Turnbull
ee2750224a Updated CHANGELOG.md, [skip ci] 2021-01-09 18:03:21 -08:00
21 changed files with 1033 additions and 679 deletions

View File

@@ -146,6 +146,16 @@
"contributions": [
"research"
]
},
{
"login": "narensankar0529",
"name": "narensankar0529",
"avatar_url": "https://avatars3.githubusercontent.com/u/74054766?v=4",
"profile": "https://github.com/narensankar0529",
"contributions": [
"bug",
"userTesting"
]
}
],
"contributorsPerLine": 7,

View File

@@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file. Dates are d
Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
#### [v0.39.15](https://github.com/RhetTbull/osxphotos/compare/v0.39.13...v0.39.15)
> 11 January 2021
- Completed implementation of --jpeg-ext, fixed --dry-run, closes #330, #346 [`#330`](https://github.com/RhetTbull/osxphotos/issues/330)
- Added --jpeg-ext, implements #330 [`55c088e`](https://github.com/RhetTbull/osxphotos/commit/55c088eea2ddecb14e362221da9e2a7c0f403780)
#### [v0.39.13](https://github.com/RhetTbull/osxphotos/compare/v0.39.12...v0.39.13)
> 10 January 2021
- Fixed leaky memory in PhotoKit, issue #276 [`db1947d`](https://github.com/RhetTbull/osxphotos/commit/db1947dd1e3d47a487eeb68a5ceb5f7098f1df10)
#### [v0.39.12](https://github.com/RhetTbull/osxphotos/compare/v0.39.11...v0.39.12)
> 9 January 2021

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2019 Rhet Turnbull
Copyright (c) 2019-2021 Rhet Turnbull
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -3,7 +3,7 @@
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![tests](https://github.com/RhetTbull/osxphotos/workflows/Tests/badge.svg)](https://github.com/RhetTbull/osxphotos/workflows/Tests/badge.svg)
<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
[![All Contributors](https://img.shields.io/badge/all_contributors-15-orange.svg?style=flat)](#contributors)
[![All Contributors](https://img.shields.io/badge/all_contributors-16-orange.svg?style=flat)](#contributors)
<!-- ALL-CONTRIBUTORS-BADGE:END -->
OSXPhotos provides the ability to interact with and query Apple's Photos.app library on macOS. You can query the Photos library database — for example, file name, file path, and metadata such as keywords/tags, persons/faces, albums, etc. You can also easily export both the original and edited photos.
@@ -112,6 +112,7 @@ Options:
-h, --help Show this message and exit.
Commands:
about Print information about osxphotos including license.
albums Print out albums found in the Photos library.
dump Print list of all photos & associated info from the Photos...
export Export photos from the Photos database.
@@ -295,6 +296,10 @@ Options:
--export-as-hardlink Hardlink files instead of copying them.
Cannot be used with --exiftool which creates
copies of the files with embedded EXIF data.
Note: on APFS volumes, files are cloned when
exporting giving many of the same advantages
as hardlinks without having to use --export-
as-hardlink.
--touch-file Sets the file's modification time to match
photo date.
--overwrite Overwrite existing files. Default behavior
@@ -489,6 +494,15 @@ Options:
do not include an extension in the FILENAME
template. See below for additional details
on templating system.
--jpeg-ext EXTENSION Specify file extension for JPEG files.
Photos uses .jpeg for edited images but many
images are imported with .jpg or .JPG which
can result in multiple different extensions
used for JPEG files upon export. Use --jpg-
ext to specify a single extension to use for
all exported JPEG images. Valid values are
jpeg, jpg, JPEG, JPG; e.g. '--jpg-ext jpg'
to use '.jpg' for all JPEGs.
--strip Optionally strip leading and trailing
whitespace from any rendered templates. For
example, if --filename template is "{title,}
@@ -2593,6 +2607,7 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
</tr>
<tr>
<td align="center"><a href="https://github.com/Rott-Apple"><img src="https://avatars1.githubusercontent.com/u/67875570?v=4?s=75" width="75px;" alt=""/><br /><sub><b>Rott-Apple</b></sub></a><br /><a href="#research-Rott-Apple" title="Research">🔬</a></td>
<td align="center"><a href="https://github.com/narensankar0529"><img src="https://avatars3.githubusercontent.com/u/74054766?v=4?s=75" width="75px;" alt=""/><br /><sub><b>narensankar0529</b></sub></a><br /><a href="https://github.com/RhetTbull/osxphotos/issues?q=author%3Anarensankar0529" title="Bug reports">🐛</a> <a href="#userTesting-narensankar0529" title="User Testing">📓</a></td>
</tr>
</table>

View File

@@ -97,6 +97,7 @@ Alternatively, you can also run the command line utility like this: ``python3 -m
-h, --help Show this message and exit.
Commands:
about Print information about osxphotos including license.
albums Print out albums found in the Photos library.
dump Print list of all photos & associated info from the Photos...
export Export photos from the Photos database.

File diff suppressed because it is too large Load Diff

View File

@@ -5,6 +5,8 @@ Constants used by osxphotos
import os.path
from datetime import datetime
OSXPHOTOS_URL = "https://github.com/RhetTbull/osxphotos"
# Time delta: add this to Photos times to get unix time
# Apple Epoch is Jan 1, 2001
TIME_DELTA = (datetime(2001, 1, 1, 0, 0) - datetime(1970, 1, 1, 0, 0)).total_seconds()
@@ -58,8 +60,16 @@ _DB_TABLE_NAMES = {
},
}
# which major version operating systems have been tested
_TESTED_OS_VERSIONS = ["12", "13", "14", "15", "16"]
# which version operating systems have been tested
_TESTED_OS_VERSIONS = [
("10", "12"),
("10", "13"),
("10", "14"),
("10", "15"),
("10", "16"),
("11", "0"),
("11", "1"),
]
# Photos 5 has persons who are empty string if unidentified face
_UNKNOWN_PERSON = "_UNKNOWN_"

View File

@@ -1,5 +1,3 @@
""" version info """
__version__ = "0.39.13"
__version__ = "0.39.17"

View File

@@ -60,6 +60,11 @@ class FileUtilABC(ABC):
def convert_to_jpeg(cls, src_file, dest_file, compression_quality=1.0):
pass
@classmethod
@abstractmethod
def rename(cls, src, dest):
pass
class FileUtilMacOS(FileUtilABC):
""" Various file utilities """
@@ -201,6 +206,21 @@ class FileUtilMacOS(FileUtilABC):
src_file, dest_file, compression_quality=compression_quality
)
@classmethod
def rename(cls, src, dest):
""" Copy src to dest
Args:
src: path to source file
dest: path to destination file
Returns:
Name of renamed file (dest)
"""
os.rename(str(src), str(dest))
return dest
@staticmethod
def _sig(st):
""" return tuple of (mode, size, mtime) of file based on os.stat
@@ -266,3 +286,7 @@ class FileUtilNoOp(FileUtil):
@classmethod
def convert_to_jpeg(cls, src_file, dest_file, compression_quality=1.0):
cls.verbose(f"convert_to_jpeg: {src_file}, {dest_file}, {compression_quality}")
@classmethod
def rename(cls, src, dest):
cls.verbose(f"rename: {src}, {dest}")

View File

@@ -49,7 +49,7 @@ from ..photokit import (
PhotoKitFetchFailed,
PhotoLibrary,
)
from ..utils import dd_to_dms_str, findfiles, noop
from ..utils import dd_to_dms_str, findfiles, noop, get_preferred_uti_extension
class ExportError(Exception):
@@ -311,6 +311,34 @@ def _check_export_suffix(src, dest, edited):
)
# not a class method, don't import into PhotoInfo
def rename_jpeg_files(files, jpeg_ext, fileutil):
""" rename any jpeg files in files so that extension matches jpeg_ext
Args:
files: list of file paths
jpeg_ext: extension to use for jpeg files found in files, e.g. "jpg"
fileutil: a FileUtil object
Returns:
list of files with updated names
Note: If non-jpeg files found, they will be ignore and returned in the return list
"""
jpeg_ext = "." + jpeg_ext
jpegs = [".jpeg", ".jpg"]
new_files = []
for file in files:
path = pathlib.Path(file)
if path.suffix.lower() in jpegs and path.suffix != jpeg_ext:
new_file = path.parent / (path.stem + jpeg_ext)
fileutil.rename(file, new_file)
new_files.append(new_file)
else:
new_files.append(file)
return new_files
def export(
self,
dest,
@@ -437,6 +465,7 @@ def export2(
exiftool_flags=None,
merge_exif_keywords=False,
merge_exif_persons=False,
jpeg_ext=None,
):
"""export photo, like export but with update and dry_run options
dest: must be valid destination path or exception raised
@@ -488,6 +517,7 @@ def export2(
exiftool_flags: optional list of flags to pass to exiftool when using exiftool option, e.g ["-m", "-F"]
merge_exif_keywords: boolean; if True, merged keywords found in file's exif data (requires exiftool)
merge_exif_persons: boolean; if True, merged persons found in file's exif data (requires exiftool)
jpeg_ext: if set, will use this value for extension on jpegs converted to jpeg with convert_to_jpeg; if not set, uses jpeg; do not include the leading "."
Returns: ExportResults class
ExportResults has attributes:
@@ -576,7 +606,8 @@ def export2(
if convert_to_jpeg and self.isphoto and uti != "public.jpeg":
# not a jpeg but will convert to jpeg upon export so fix file extension
fname_new = pathlib.Path(fname)
fname = str(fname_new.parent / f"{fname_new.stem}.jpeg")
ext = "." + jpeg_ext if jpeg_ext else ".jpeg"
fname = str(fname_new.parent / f"{fname_new.stem}{ext}")
else:
# nothing to convert
convert_to_jpeg = False
@@ -746,6 +777,8 @@ def export2(
)
all_results += results
else:
# TODO: move this big if/else block to separate functions
# e.g. _export_with_photos_export or such
# use_photo_export
# export live_photo .mov file?
live_photo = True if live_photo and self.live_photo else False
@@ -760,7 +793,10 @@ def export2(
else:
# didn't get passed a filename, add _edited
filestem = f"{dest.stem}{edited_identifier}"
dest = dest.parent / f"{filestem}.jpeg"
uti = self.uti_edited if edited and self.uti_edited else self.uti
ext = get_preferred_uti_extension(uti)
dest = dest.parent / f"{filestem}{ext}"
if use_photokit:
photolib = PhotoLibrary()
photo = None
@@ -783,13 +819,17 @@ def export2(
)
)
if photo:
try:
exported = photo.export(
dest.parent, dest.name, version=PHOTOS_VERSION_CURRENT
)
all_results.exported.extend(exported)
except Exception as e:
all_results.error.append((str(dest), e))
if not dry_run:
try:
exported = photo.export(
dest.parent, dest.name, version=PHOTOS_VERSION_CURRENT
)
all_results.exported.extend(exported)
except Exception as e:
all_results.error.append((str(dest), e))
else:
# dry_run, don't actually export
all_results.exported.append(str(dest))
else:
try:
exported = _export_photo_uuid_applescript(
@@ -824,13 +864,17 @@ def export2(
photo = [p for p in bursts if p.uuid.startswith(self.uuid)]
photo = photo[0] if photo else None
if photo:
try:
exported = photo.export(
dest.parent, dest.name, version=PHOTOS_VERSION_ORIGINAL
)
all_results.exported.extend(exported)
except Exception as e:
all_results.error.append((str(dest), e))
if not dry_run:
try:
exported = photo.export(
dest.parent, dest.name, version=PHOTOS_VERSION_ORIGINAL
)
all_results.exported.extend(exported)
except Exception as e:
all_results.error.append((str(dest), e))
else:
# dry_run, don't actually export
all_results.exported.append(str(dest))
else:
try:
exported = _export_photo_uuid_applescript(
@@ -848,6 +892,13 @@ def export2(
except ExportError as e:
all_results.error.append((str(dest), e))
if all_results.exported:
if jpeg_ext:
# use_photos_export (both PhotoKit and AppleScript) don't use the
# file extension provided (instead they use extension for UTI)
# so if jpeg_ext is set, rename any non-conforming jpegs
all_results.exported = rename_jpeg_files(
all_results.exported, jpeg_ext, fileutil
)
if touch_file:
for exported_file in all_results.exported:
all_results.touched.append(exported_file)
@@ -856,9 +907,6 @@ def export2(
if update:
all_results.new.extend(all_results.exported)
# else:
# all_results.error.append((str(dest), f"Error exporting photo {self.uuid} to {dest} with use_photos_export"))
# export metadata
sidecars = []
sidecar_json_files_skipped = []
@@ -1766,3 +1814,4 @@ def _write_sidecar(self, filename, sidecar_str):
f = open(filename, "w")
f.write(sidecar_str)
f.close()

View File

@@ -85,12 +85,12 @@ class PhotosDB:
# Check OS version
system = platform.system()
(_, major, _) = _get_os_version()
if system != "Darwin" or (major not in _TESTED_OS_VERSIONS):
(ver, major, _) = _get_os_version()
if system != "Darwin" or ((ver, major) not in _TESTED_OS_VERSIONS):
logging.warning(
f"WARNING: This module has only been tested with MacOS 10."
f"[{', '.join(_TESTED_OS_VERSIONS)}]: "
f"you have {system}, OS version: {major}"
f"WARNING: This module has only been tested with macOS versions "
f"[{', '.join(f'{v}.{m}' for (v, m) in _TESTED_OS_VERSIONS)}]: "
f"you have {system}, OS version: {ver}.{major}"
)
if verbose is None:
@@ -1160,9 +1160,9 @@ class PhotosDB:
self._dbphotos[uuid]["original_orientation"] = row[38]
self._dbphotos[uuid]["original_filesize"] = row[39]
# visibility state
self._dbphotos[uuid]["visibility_state"] = row[42]
self._dbphotos[uuid]["visible"] = row[42] == 1
# visibility state
self._dbphotos[uuid]["visibility_state"] = row[42]
self._dbphotos[uuid]["visible"] = row[42] == 1
# import session not yet handled for Photos 4
self._dbphotos[uuid]["import_session"] = None

File diff suppressed because one or more lines are too long

View File

@@ -1117,9 +1117,14 @@ def test_photosinfo_repr():
def test_from_to_date():
import os
import time
import osxphotos
import datetime as dt
os.environ["TZ"] = "US/Pacific"
time.tzset()
photosdb = osxphotos.PhotosDB(PHOTOS_DB)
photos = photosdb.photos(from_date=dt.datetime(2018, 10, 28))

View File

@@ -1119,6 +1119,11 @@ def test_photosinfo_repr():
def test_from_to_date():
import osxphotos
import datetime as dt
import os
import time
os.environ["TZ"] = "US/Pacific"
time.tzset()
photosdb = osxphotos.PhotosDB(PHOTOS_DB)

View File

@@ -763,6 +763,11 @@ def test_photosinfo_repr():
def test_from_to_date():
import osxphotos
import datetime as dt
import os
import time
os.environ["TZ"] = "US/Pacific"
time.tzset()
photosdb = osxphotos.PhotosDB(PHOTOS_DB)

View File

@@ -771,6 +771,11 @@ def test_photosinfo_repr():
def test_from_to_date():
import osxphotos
import datetime as dt
import os
import time
os.environ["TZ"] = "US/Pacific"
time.tzset()
photosdb = osxphotos.PhotosDB(PHOTOS_DB)

View File

@@ -1028,6 +1028,11 @@ def test_photosinfo_repr():
def test_from_to_date():
import osxphotos
import datetime as dt
import os
import time
os.environ["TZ"] = "US/Pacific"
time.tzset()
photosdb = osxphotos.PhotosDB(PHOTOS_DB)

View File

@@ -565,6 +565,14 @@ UUID_NO_LIKES = [
"1C1C8F1F-826B-4A24-B1CB-56628946A834",
]
UUID_JPEGS_DICT = {
"4D521201-92AC-43E5-8F7C-59BC41C37A96": ["IMG_1997", "JPG"],
"E9BC5C36-7CD1-40A1-A72B-8B8FAC227D51": ["wedding", "jpg"],
"E2078879-A29C-4D6F-BACB-E3BBE6C3EB91": ["screenshot-really-a-png", "jpeg"],
}
UUID_HEIC = {"7783E8E6-9CAC-40F3-BE22-81FB7051C266": "IMG_3092"}
def modify_file(filename):
""" appends data to a file to modify it """
@@ -684,6 +692,17 @@ def test_osxphotos_help_3():
assert "Invalid command: foo" in result.output
def test_about():
""" Test about """
from osxphotos.__main__ import about
runner = CliRunner()
cwd = os.getcwd()
result = runner.invoke(about, [])
assert result.exit_code == 0
assert "MIT License" in result.output
def test_query_uuid():
import json
import os
@@ -1568,7 +1587,7 @@ def test_export_convert_to_jpeg():
files = glob.glob("*")
assert sorted(files) == sorted(CLI_EXPORT_FILENAMES_CONVERT_TO_JPEG)
large_file = pathlib.Path(CLI_EXPORT_CONVERT_TO_JPEG_LARGE_FILE)
assert large_file.stat().st_size > 10000000
assert large_file.stat().st_size > 7000000
@pytest.mark.skipif(
@@ -5238,3 +5257,77 @@ def test_export_xattr_template():
assert sorted(md.keywords) == sorted(expected)
assert md.comment == CLI_FINDER_TAGS[uuid]["XMP:Title"]
def test_export_jpeg_ext():
""" test --jpeg-ext """
import glob
import os
import os.path
from osxphotos.__main__ import export
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
for uuid, fileinfo in UUID_JPEGS_DICT.items():
result = runner.invoke(
export, [os.path.join(cwd, PHOTOS_DB_15_7), ".", "-V", "--uuid", uuid]
)
assert result.exit_code == 0
files = glob.glob("*")
filename, ext = fileinfo
assert f"{filename}.{ext}" in files
for jpeg_ext in ["jpg", "JPG", "jpeg", "JPEG"]:
with runner.isolated_filesystem():
for uuid, fileinfo in UUID_JPEGS_DICT.items():
result = runner.invoke(
export,
[
os.path.join(cwd, PHOTOS_DB_15_7),
".",
"-V",
"--uuid",
uuid,
"--jpeg-ext",
jpeg_ext,
],
)
assert result.exit_code == 0
files = glob.glob("*")
filename, ext = fileinfo
assert f"{filename}.{jpeg_ext}" in files
@pytest.mark.skipif(
"OSXPHOTOS_TEST_CONVERT" not in os.environ,
reason="Skip if running in Github actions, no GPU.",
)
def test_export_jpeg_ext_convert_to_jpeg():
""" test --jpeg-ext with --convert-to-jpeg """
import glob
import os
import os.path
from osxphotos.__main__ import export
runner = CliRunner()
cwd = os.getcwd()
# pylint: disable=not-context-manager
with runner.isolated_filesystem():
for uuid, filename in UUID_HEIC.items():
result = runner.invoke(
export,
[
os.path.join(cwd, PHOTOS_DB_15_7),
".",
"-V",
"--uuid",
uuid,
"--convert-to-jpeg",
"--jpeg-ext",
"jpg",
],
)
assert result.exit_code == 0
files = glob.glob("*")
assert f"{filename}.jpg" in files

View File

@@ -107,3 +107,21 @@ def test_convert_to_jpeg_quality():
assert FileUtil.convert_to_jpeg(imgfile, outfile, compression_quality=0.1)
assert outfile.is_file()
assert outfile.stat().st_size < 1000000
def test_rename_file():
# rename file with valid src, dest
import pathlib
import tempfile
from osxphotos.fileutil import FileUtil
temp_dir = tempfile.TemporaryDirectory(prefix="osxphotos_")
src = "tests/test-images/wedding.jpg"
dest = f"{temp_dir.name}/foo.jpg"
dest2 = f"{temp_dir.name}/bar.jpg"
FileUtil.copy(src, dest)
result = FileUtil.rename(dest, dest2)
assert result
assert pathlib.Path(dest2).exists()
assert not pathlib.Path(dest).exists()

18
utils/README.md Normal file
View File

@@ -0,0 +1,18 @@
# Utils
These are various utilities used in my development workflow. They may or may not be useful to you if you're working on osxphotos. If using the AppleScripts to get data from Photos, I highly recommend the excellent [FastScripts](https://redsweater.com/fastscripts/) from Red Sweater Software.
## Files
|File | Description |
|-----|-------------|
|build_help_table.py| Builds the template substitutions table used in main README.md |
|check_uuid.py| Use with output file created by dump_photo_info.scpt to check ouput of osxphotos vs what Photos reports|
|copy_uuid_to_clipboard.applescript| Copy UUID of selected photo in Photos to the Clipboard|
|dump_photo_info.applescript| Dumps UUID and other info about every photo in Photos.app to a test file; see check_uuid.py|
|dump_photo_info.scpt| Compiled version of dump_photo_info.applescript|
|gen_face_test_data.py| Generate test data for test_faceinfo.py|
|generate_search_info_test_data.py | Create the test data needed for test_search_info_10_15_7.py|
|get_photo_info.applescript| Displays UUID and other info about selected photos, useful for debugging|
|get_photo_info.scpt| Compiled version of above|
|write_uuid_to_file.applescript| Writes the UUIDs of selected images in Photos to a text file; can generate input for --uuid-from-file|

View File

@@ -0,0 +1,20 @@
-- Copies UUID of selected photo to the clipboard, if more than one selection, copies uuid from the last item
-- Useful for debugging with osxphotos
tell application "Photos"
set uuid to ""
set theSelection to selection
repeat with theItem in theSelection
set uuid to ((id of theItem) as text)
set oldDelimiter to AppleScript's text item delimiters
set AppleScript's text item delimiters to "/"
set theTextItems to every text item of uuid
set uuid to first item of theTextItems
set AppleScript's text item delimiters to oldDelimiter
end repeat
set the clipboard to uuid
end tell