Fix for --load-config, #643

* Initial refactoring of cli.py

* Renamed cli_help

* Refactored all cli commands

* Dropped support for 3.7

* Added test for export with --min-size

* Version bump

* Fixed python version

.github/workflows/tests.yml

@@ -10,7 +10,7 @@ jobs:
       max-parallel: 4
       matrix:
         os: [macos-10.15]
-        python-version: [3.7, 3.8, 3.9]
+        python-version: ['3.8', '3.9', '3.10']
 
     steps:
     - uses: actions/checkout@v1

CHANGELOG.md

@@ -4,6 +4,74 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [v0.47.1](https://github.com/RhetTbull/osxphotos/compare/v0.47.0...v0.47.1)
+
+> 26 February 2022
+
+- Fixed entry point [`d275367`](https://github.com/RhetTbull/osxphotos/commit/d2753672f36a0c84b7be143a47cc85cd6c99cb6d)
+
+#### [v0.47.0](https://github.com/RhetTbull/osxphotos/compare/v0.46.6...v0.47.0)
+
+> 26 February 2022
+
+- CLI refactor [`#642`](https://github.com/RhetTbull/osxphotos/pull/642)
+- Updated docs [skip ci] [`6fae979`](https://github.com/RhetTbull/osxphotos/commit/6fae97906124c9284e382170e20c8ab9999105b0)
+- Fixed 3.10 in yaml [`3704fc4`](https://github.com/RhetTbull/osxphotos/commit/3704fc4a23e83ff2d16d6d221fb6c752dabcedca)
+- Dropped 3.7 [`7883fc1`](https://github.com/RhetTbull/osxphotos/commit/7883fc1911057df9a4c596375b498e85a73c1bec)
+
+#### [v0.46.6](https://github.com/RhetTbull/osxphotos/compare/v0.46.5...v0.46.6)
+
+> 26 February 2022
+
+- Updated tests [`43e1cb1`](https://github.com/RhetTbull/osxphotos/commit/43e1cb18cc65b1abe1f49b464563e816b2ed1cff)
+- Updated docs [skip ci] [`3a990e3`](https://github.com/RhetTbull/osxphotos/commit/3a990e39971d838e52d5f19bf28b8253c4c7b811)
+- Bug fix for bitmath types in saved config [`26f916e`](https://github.com/RhetTbull/osxphotos/commit/26f916e4cbf4f28154c47aa2de1fdbc0aebc65b3)
+
+#### [v0.46.5](https://github.com/RhetTbull/osxphotos/compare/v0.46.4...v0.46.5)
+
+> 24 February 2022
+
+- Updated tested versions [`4d1b1db`](https://github.com/RhetTbull/osxphotos/commit/4d1b1db2a7cf34afaa2dc5dbebc69021ff77964f)
+
+#### [v0.46.4](https://github.com/RhetTbull/osxphotos/compare/v0.46.1...v0.46.4)
+
+> 24 February 2022
+
+- Removed debug code from exiftool, fixed #641 [`#641`](https://github.com/RhetTbull/osxphotos/issues/641)
+- Added debug output to exiftool [`39ba17d`](https://github.com/RhetTbull/osxphotos/commit/39ba17dd1cb4d8a61ab4dc8d5cff12ff9871eee0)
+- Fixed export of bursts with --uuid and --selected, #640 [`5b66962`](https://github.com/RhetTbull/osxphotos/commit/5b66962ac1bc1f48106fb8eeb600e6010088dc3b)
+- Added --sql command to exportdb [`c8ee679`](https://github.com/RhetTbull/osxphotos/commit/c8ee6797999af954c32e96ac3799a19002f4f0fe)
+- Updated docs [skip ci] [`2966c9a`](https://github.com/RhetTbull/osxphotos/commit/2966c9a60fc828afdf34263b759159a3ade31897)
+- Updated debug info [`6cb7ded`](https://github.com/RhetTbull/osxphotos/commit/6cb7dedd9be53d2c62489125fc44b9f4dccfb7ae)
+
+#### [v0.46.1](https://github.com/RhetTbull/osxphotos/compare/v0.46.0...v0.46.1)
+
+> 21 February 2022
+
+- Added --ramdb option [`#639`](https://github.com/RhetTbull/osxphotos/pull/639)
+
+#### [v0.46.0](https://github.com/RhetTbull/osxphotos/compare/v0.45.12...v0.46.0)
+
+> 21 February 2022
+
+- Exportdb refactor [`#638`](https://github.com/RhetTbull/osxphotos/pull/638)
+- Updated docs [skip ci] [`5290fae`](https://github.com/RhetTbull/osxphotos/commit/5290fae2e0ad062750348aedfee4feaf7b2e769f)
+
+#### [v0.45.12](https://github.com/RhetTbull/osxphotos/compare/v0.45.11...v0.45.12)
+
+> 14 February 2022
+
+- Allow multiple characters as path_sep, #634 [`d8204e6`](https://github.com/RhetTbull/osxphotos/commit/d8204e65eb740cece468ef021cbdf45d896d954e)
+- Added --debug and crash reporter to export, #628 [`060729c`](https://github.com/RhetTbull/osxphotos/commit/060729c4c4255651c6ee8149989d9de541d0a6aa)
+- Added crash_reporter.py [`9c26e55`](https://github.com/RhetTbull/osxphotos/commit/9c26e5519b2d48f3a0ae80d1cc4a765c12b62d40)
+
+#### [v0.45.11](https://github.com/RhetTbull/osxphotos/compare/v0.45.10...v0.45.11)
+
+> 13 February 2022
+
+- beta fix for #633, fix face regions in exiftool [`afbda03`](https://github.com/RhetTbull/osxphotos/commit/afbda030bce87f914445ebbced3f0e110e2e203b)
+- Updated docs [skip ci] [`65d51ab`](https://github.com/RhetTbull/osxphotos/commit/65d51ab1290e7c7804021e24829b93f5dce81245)
+
 #### [v0.45.10](https://github.com/RhetTbull/osxphotos/compare/v0.45.9...v0.45.10)
 
 > 12 February 2022

README.md

@@ -68,7 +68,7 @@ Only works on macOS (aka Mac OS X). Tested on macOS Sierra (10.12.6) through mac
 
 This package will read Photos databases for any supported version on any supported macOS version.  E.g. you can read a database created with Photos 5.0 on MacOS 10.15 on a machine running macOS 10.12 and vice versa.
 
-Requires python >= `3.7`. 
+Requires python >= `3.8`. 
 
 
 ## Installation 
@@ -1173,6 +1173,11 @@ Options:
                                   '.osxphotos_export.db' in the export
                                   directory.  If --exportdb is specified, it
                                   will be saved to the specified file.
+  --ramdb                         Copy export database to memory during export;
+                                  may improve performance when exporting over a
+                                  network or slow disk but could result in
+                                  losing update state information if the program
+                                  is interrupted or crashes.
   --load-config <config file path>
                                   Load options from file as written with --save-
                                   config. This allows you to save a complex
@@ -1191,7 +1196,7 @@ Options:
   --config-only                   If specified, saves the config file but does
                                   not export any files; must be used with
                                   --save-config.
-  --help                          Show this message and exit.
+  -h, --help                      Show this message and exit.
 
 ** Export **
 
@@ -1736,7 +1741,7 @@ Substitution                    Description
 {lf}                            A line feed: '\n', alias for {newline}
 {cr}                            A carriage return: '\r'
 {crlf}                          a carriage return + line feed: '\r\n'
-{osxphotos_version}             The osxphotos version, e.g. '0.46.0'
+{osxphotos_version}             The osxphotos version, e.g. '0.47.1'
 {osxphotos_cmd_line}            The full command line used to run osxphotos
 
 The following substitutions may result in multiple values. Thus if specified for
@@ -3640,7 +3645,7 @@ The following template field substitutions are availabe for use the templating s
 |{lf}|A line feed: '\n', alias for {newline}|
 |{cr}|A carriage return: '\r'|
 |{crlf}|a carriage return + line feed: '\r\n'|
-|{osxphotos_version}|The osxphotos version, e.g. '0.46.0'|
+|{osxphotos_version}|The osxphotos version, e.g. '0.47.1'|
 |{osxphotos_cmd_line}|The full command line used to run osxphotos|
 |{album}|Album(s) photo is contained in|
 |{folder_album}|Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder|

README.rst

@@ -23,7 +23,7 @@ If you have access to macOS 12 / Monterey beta and would like to help ensure osx
 This package will read Photos databases for any supported version on any supported macOS version.  
 E.g. you can read a database created with Photos 5.0 on MacOS 10.15 on a machine running macOS 10.12 and vice versa.
 
-Requires python >= ``3.7``. 
+Requires python >= ``3.8``. 
 
 Installation
 ------------

cli.py

@@ -12,7 +12,7 @@
     
 """
 
-from osxphotos.cli import cli
+from osxphotos.cli.cli import cli_main
 
 if __name__ == "__main__":
-    cli()
+    cli_main()

dev_requirements.txt

@@ -3,7 +3,7 @@ m2r2
 pdbpp
 pyinstaller==4.4
 pytest-mock
-pytest==6.2.4
+pytest==7.0.1
 Sphinx
 sphinx_click
 sphinx_rtd_theme

docs/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 5b6236594d7900f08d9a1afda487bf3c
+config: bc3dce8a14bcd1b0c8a34e4d16f0011f
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_modules/index.html

@@ -5,7 +5,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Overview: module code &#8212; osxphotos 0.46.0 documentation</title>
+    <title>Overview: module code &#8212; osxphotos 0.47.1 documentation</title>
     <link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="../_static/alabaster.css" />
     <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>

docs/_modules/osxphotos/photoinfo.html

@@ -5,7 +5,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>osxphotos.photoinfo &#8212; osxphotos 0.46.0 documentation</title>
+    <title>osxphotos.photoinfo &#8212; osxphotos 0.46.4 documentation</title>
     <link rel="stylesheet" type="text/css" href="../../_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="../../_static/alabaster.css" />
     <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
@@ -87,7 +87,7 @@
 <span class="kn">from</span> <span class="nn">.searchinfo</span> <span class="kn">import</span> <span class="n">SearchInfo</span>
 <span class="kn">from</span> <span class="nn">.text_detection</span> <span class="kn">import</span> <span class="n">detect_text</span>
 <span class="kn">from</span> <span class="nn">.uti</span> <span class="kn">import</span> <span class="n">get_preferred_uti_extension</span><span class="p">,</span> <span class="n">get_uti_for_extension</span>
-<span class="kn">from</span> <span class="nn">.utils</span> <span class="kn">import</span> <span class="n">_debug</span><span class="p">,</span> <span class="n">_get_resource_loc</span><span class="p">,</span> <span class="n">list_directory</span>
+<span class="kn">from</span> <span class="nn">.utils</span> <span class="kn">import</span> <span class="n">_debug</span><span class="p">,</span> <span class="n">_get_resource_loc</span><span class="p">,</span> <span class="n">list_directory</span><span class="p">,</span> <span class="n">_debug</span>
 
 <span class="n">__all__</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;PhotoInfo&quot;</span><span class="p">,</span> <span class="s2">&quot;PhotoInfoNone&quot;</span><span class="p">]</span>
 
@@ -621,7 +621,7 @@
     <span class="nd">@property</span>
     <span class="k">def</span> <span class="nf">ismissing</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
         <span class="sd">&quot;&quot;&quot;returns true if photo is missing from disk (which means it&#39;s not been downloaded from iCloud)</span>
-<span class="sd">        </span>
+
 <span class="sd">        NOTE:   the photos.db database uses an asynchrounous write-ahead log so changes in Photos</span>
 <span class="sd">                do not immediately get written to disk. In particular, I&#39;ve noticed that downloading</span>
 <span class="sd">                an image from the cloud does not force the database to be updated until something else</span>
@@ -1818,7 +1818,6 @@
 <h3>Navigation</h3>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="../../cli.html">osxphotos command line interface (CLI)</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../modules.html">osxphotos</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../../reference.html">osxphotos package</a></li>
 </ul>
 

docs/_modules/osxphotos/photosdb/photosdb.html

@@ -5,7 +5,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>osxphotos.photosdb.photosdb &#8212; osxphotos 0.46.0 documentation</title>
+    <title>osxphotos.photosdb.photosdb &#8212; osxphotos 0.46.6 documentation</title>
     <link rel="stylesheet" type="text/css" href="../../../_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="../../../_static/alabaster.css" />
     <script data-url_root="../../../" id="documentation_options" src="../../../_static/documentation_options.js"></script>
@@ -3312,27 +3312,6 @@
         <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">to_time</span><span class="p">:</span>
             <span class="n">photos</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span> <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">&lt;=</span> <span class="n">options</span><span class="o">.</span><span class="n">to_time</span><span class="p">]</span>
 
-        <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">:</span>
-            <span class="c1"># add the burst_photos to the export set</span>
-            <span class="n">photos_burst</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span> <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">burst</span><span class="p">]</span>
-            <span class="k">for</span> <span class="n">burst</span> <span class="ow">in</span> <span class="n">photos_burst</span><span class="p">:</span>
-                <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">missing_bursts</span><span class="p">:</span>
-                    <span class="c1"># include burst photos that are missing</span>
-                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">)</span>
-                <span class="k">else</span><span class="p">:</span>
-                    <span class="c1"># don&#39;t include missing burst images (these can&#39;t be downloaded with AppleScript)</span>
-                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">p</span><span class="o">.</span><span class="n">ismissing</span><span class="p">])</span>
-
-            <span class="c1"># remove duplicates as each burst photo in the set that&#39;s selected would</span>
-            <span class="c1"># result in the entire set being added above</span>
-            <span class="c1"># can&#39;t use set() because PhotoInfo not hashable</span>
-            <span class="n">seen_uuids</span> <span class="o">=</span> <span class="p">{}</span>
-            <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span><span class="p">:</span>
-                <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">uuid</span> <span class="ow">in</span> <span class="n">seen_uuids</span><span class="p">:</span>
-                    <span class="k">continue</span>
-                <span class="n">seen_uuids</span><span class="p">[</span><span class="n">p</span><span class="o">.</span><span class="n">uuid</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span>
-            <span class="n">photos</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">seen_uuids</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>
-
         <span class="k">if</span> <span class="n">name</span><span class="p">:</span>
             <span class="c1"># search filename fields for text</span>
             <span class="c1"># if more than one, find photos with all title values in filename</span>
@@ -3483,6 +3462,28 @@
             <span class="k">for</span> <span class="n">function</span> <span class="ow">in</span> <span class="n">options</span><span class="o">.</span><span class="n">function</span><span class="p">:</span>
                 <span class="n">photos</span> <span class="o">=</span> <span class="n">function</span><span class="p">[</span><span class="mi">0</span><span class="p">](</span><span class="n">photos</span><span class="p">)</span>
 
+        <span class="c1"># burst should be checked last, ref #640</span>
+        <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">:</span>
+            <span class="c1"># add the burst_photos to the export set</span>
+            <span class="n">photos_burst</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span> <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">burst</span><span class="p">]</span>
+            <span class="k">for</span> <span class="n">burst</span> <span class="ow">in</span> <span class="n">photos_burst</span><span class="p">:</span>
+                <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">missing_bursts</span><span class="p">:</span>
+                    <span class="c1"># include burst photos that are missing</span>
+                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">)</span>
+                <span class="k">else</span><span class="p">:</span>
+                    <span class="c1"># don&#39;t include missing burst images (these can&#39;t be downloaded with AppleScript)</span>
+                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">p</span><span class="o">.</span><span class="n">ismissing</span><span class="p">])</span>
+
+            <span class="c1"># remove duplicates as each burst photo in the set that&#39;s selected would</span>
+            <span class="c1"># result in the entire set being added above</span>
+            <span class="c1"># can&#39;t use set() because PhotoInfo not hashable</span>
+            <span class="n">seen_uuids</span> <span class="o">=</span> <span class="p">{}</span>
+            <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span><span class="p">:</span>
+                <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">uuid</span> <span class="ow">in</span> <span class="n">seen_uuids</span><span class="p">:</span>
+                    <span class="k">continue</span>
+                <span class="n">seen_uuids</span><span class="p">[</span><span class="n">p</span><span class="o">.</span><span class="n">uuid</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span>
+            <span class="n">photos</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">seen_uuids</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>
+
         <span class="k">return</span> <span class="n">photos</span></div>
 
 <div class="viewcode-block" id="PhotosDB.execute"><a class="viewcode-back" href="../../../reference.html#osxphotos.PhotosDB.execute">[docs]</a>    <span class="k">def</span> <span class="nf">execute</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sql</span><span class="p">):</span>
@@ -3568,7 +3569,6 @@
 <h3>Navigation</h3>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="../../../cli.html">osxphotos command line interface (CLI)</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../../modules.html">osxphotos</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../../../reference.html">osxphotos package</a></li>
 </ul>
 

docs/_static/documentation_options.js

@@ -1,6 +1,6 @@
 var DOCUMENTATION_OPTIONS = {
     URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
-    VERSION: '0.46.0',
+    VERSION: '0.47.1',
     LANGUAGE: 'None',
     COLLAPSE_INDEX: false,
     BUILDER: 'html',

docs/index.html

@@ -6,7 +6,7 @@
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
 
-    <title>Welcome to osxphotos’s documentation! &#8212; osxphotos 0.46.0 documentation</title>
+    <title>Welcome to osxphotos’s documentation! &#8212; osxphotos 0.47.1 documentation</title>
     <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
     <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
@@ -50,7 +50,7 @@ You can also easily export both the original and edited photos.</p>
 <p>If you have access to macOS 12 / Monterey beta and would like to help ensure osxphotos is compatible, please contact me via GitHub.</p>
 <p>This package will read Photos databases for any supported version on any supported macOS version.
 E.g. you can read a database created with Photos 5.0 on MacOS 10.15 on a machine running macOS 10.12 and vice versa.</p>
-<p>Requires python &gt;= <code class="docutils literal notranslate"><span class="pre">3.7</span></code>.</p>
+<p>Requires python &gt;= <code class="docutils literal notranslate"><span class="pre">3.8</span></code>.</p>
 </section>
 <section id="installation">
 <h2>Installation<a class="headerlink" href="#installation" title="Permalink to this headline">¶</a></h2>
@@ -281,32 +281,7 @@ Alternatively, you can also run the command line utility like this: <code class=
 <p>Reference full documentation on <a class="reference external" href="https://github.com/RhetTbull/osxphotos/blob/master/README.md">GitHub</a></p>
 <div class="toctree-wrapper compound">
 <ul>
-<li class="toctree-l1"><a class="reference internal" href="cli.html">osxphotos command line interface (CLI)</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="cli.html#osxphotos">osxphotos</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-about">about</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-albums">albums</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-diff">diff</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-dump">dump</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-export">export</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-help">help</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-info">info</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-install">install</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-keywords">keywords</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-labels">labels</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-list">list</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-persons">persons</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-places">places</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-query">query</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-repl">repl</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-run">run</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-snap">snap</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-tutorial">tutorial</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-uninstall">uninstall</a></li>
-<li class="toctree-l3"><a class="reference internal" href="cli.html#osxphotos-uuid">uuid</a></li>
-</ul>
-</li>
-</ul>
-</li>
+<li class="toctree-l1"><a class="reference internal" href="cli.html">osxphotos command line interface (CLI)</a></li>
 <li class="toctree-l1"><a class="reference internal" href="reference.html">osxphotos package</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="reference.html#osxphotos-module">osxphotos module</a></li>
 </ul>

docs/modules.html

@@ -6,7 +6,7 @@
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
 
-    <title>osxphotos &#8212; osxphotos 0.46.0 documentation</title>
+    <title>osxphotos &#8212; osxphotos 0.47.1 documentation</title>
     <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
     <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>

docs/reference.html

@@ -6,7 +6,7 @@
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
 
-    <title>osxphotos package &#8212; osxphotos 0.46.0 documentation</title>
+    <title>osxphotos package &#8212; osxphotos 0.47.1 documentation</title>
     <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
     <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
@@ -15,7 +15,7 @@
     <script src="_static/doctools.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
-    <link rel="prev" title="osxphotos" href="modules.html" />
+    <link rel="prev" title="osxphotos command line interface (CLI)" href="cli.html" />
    
   <link rel="stylesheet" href="_static/custom.css" type="text/css" />
   
@@ -935,7 +935,6 @@ Returns None if no associated RAW image</p>
 <h3>Navigation</h3>
 <ul class="current">
 <li class="toctree-l1"><a class="reference internal" href="cli.html">osxphotos command line interface (CLI)</a></li>
-<li class="toctree-l1"><a class="reference internal" href="modules.html">osxphotos</a></li>
 <li class="toctree-l1 current"><a class="current reference internal" href="#">osxphotos package</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="#osxphotos-module">osxphotos module</a></li>
 </ul>
@@ -946,7 +945,7 @@ Returns None if no associated RAW image</p>
 <h3>Related Topics</h3>
 <ul>
   <li><a href="index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="modules.html" title="previous chapter">osxphotos</a></li>
+      <li>Previous: <a href="cli.html" title="previous chapter">osxphotos command line interface (CLI)</a></li>
   </ul></li>
 </ul>
 </div>

docs/search.html

@@ -5,7 +5,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Search &#8212; osxphotos 0.46.0 documentation</title>
+    <title>Search &#8212; osxphotos 0.47.1 documentation</title>
     <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
     

osxphotos/__main__.py

@@ -1,6 +1,6 @@
 """Command line interface for osxphotos """
 
-from .cli import cli
+from .cli.cli import cli_main
 
 if __name__ == "__main__":
-    cli()  # pylint: disable=no-value-for-parameter
+    cli_main()

osxphotos/_constants.py

@@ -103,6 +103,8 @@ _TESTED_OS_VERSIONS = [
     ("11", "6"),
     ("12", "0"),
     ("12", "1"),
+    ("12", "2"),
+    ("12", "3"),
 ]
 
 # Photos 5 has persons who are empty string if unidentified face
@@ -231,10 +233,6 @@ DEFAULT_ORIGINAL_SUFFIX = ""
 # Default suffix to add to preview images
 DEFAULT_PREVIEW_SUFFIX = "_preview"
 
-# Colors for print CLI messages
-CLI_COLOR_ERROR = "red"
-CLI_COLOR_WARNING = "yellow"
-
 # Bit masks for --sidecar
 SIDECAR_JSON = 0x1
 SIDECAR_EXIFTOOL = 0x2
@@ -259,6 +257,7 @@ EXTENDED_ATTRIBUTE_NAMES = [
 ]
 EXTENDED_ATTRIBUTE_NAMES_QUOTED = [f"'{x}'" for x in EXTENDED_ATTRIBUTE_NAMES]
 
+
 # name of export DB
 OSXPHOTOS_EXPORT_DB = ".osxphotos_export.db"
 

osxphotos/_version.py

@@ -1,3 +1,3 @@
 """ version info """
 
-__version__ = "0.46.0"
+__version__ = "0.47.2"

osxphotos/cli/__init__.py

@@ -0,0 +1,56 @@
+"""cli package for osxphotos"""
+
+from rich.traceback import install as install_traceback
+
+from .about import about
+from .albums import albums
+from .cli import cli_main
+from .common import get_photos_db, load_uuid_from_file, set_debug
+from .debug_dump import debug_dump
+from .dump import dump
+from .export import export
+from .exportdb import exportdb
+from .grep import grep
+from .help import help
+from .info import info
+from .install_uninstall_run import install, run, uninstall
+from .keywords import keywords
+from .labels import labels
+from .list import _list_libraries, list_libraries
+from .persons import persons
+from .places import places
+from .query import query
+from .repl import repl
+from .snap_diff import diff, snap
+from .tutorial import tutorial
+from .uuid import uuid
+
+install_traceback()
+
+__all__ = [
+    "about",
+    "albums",
+    "cli_main",
+    "debug_dump",
+    "diff",
+    "dump",
+    "export",
+    "exportdb",
+    "grep",
+    "help",
+    "info",
+    "install",
+    "keywords",
+    "labels",
+    "list_libraries",
+    "list_libraries",
+    "load_uuid_from_file",
+    "persons",
+    "places",
+    "query",
+    "repl",
+    "run",
+    "snap",
+    "tutorial",
+    "uuid",
+]

osxphotos/cli/about.py

@@ -0,0 +1,66 @@
+"""about command for osxphotos CLI"""
+
+import click
+
+from osxphotos._constants import OSXPHOTOS_URL
+from osxphotos._version import __version__
+
+
+@click.command(name="about")
+@click.pass_obj
+@click.pass_context
+def about(ctx, cli_obj):
+    """Print information about osxphotos including license."""
+    license = """
+MIT License
+
+Copyright (c) 2019-2021 Rhet Turnbull
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+osxphotos uses the following 3rd party software licensed under the BSD-3-Clause License:
+Click (Copyright 2014 Pallets), ptpython (Copyright (c) 2015, Jonathan Slenders)
+
+Redistribution and use in source and binary forms, with or without modification, are 
+permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list 
+of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice, this list 
+of conditions and the following disclaimer in the documentation and/or other materials 
+provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors may be 
+used to endorse or promote products derived from this software without specific prior 
+written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR 
+IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY 
+AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER 
+OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, 
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 
+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+"""
+    click.echo(f"osxphotos, version {__version__}")
+    click.echo("")
+    click.echo(f"Source code available at: {OSXPHOTOS_URL}")
+    click.echo(license)

osxphotos/cli/albums.py

@@ -0,0 +1,42 @@
+"""albums command for osxphotos CLI"""
+
+import json
+
+import click
+import yaml
+
+import osxphotos
+
+from .common import DB_ARGUMENT, DB_OPTION, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+
+from osxphotos._constants import _PHOTOS_4_VERSION
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def albums(ctx, cli_obj, db, json_, photos_library):
+    """Print out albums found in the Photos library."""
+
+    # below needed for to make CliRunner work for testing
+    cli_db = cli_obj.db if cli_obj is not None else None
+    db = get_photos_db(*photos_library, db, cli_db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["albums"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    albums = {"albums": photosdb.albums_as_dict}
+    if photosdb.db_version > _PHOTOS_4_VERSION:
+        albums["shared albums"] = photosdb.albums_shared_as_dict
+
+    if json_ or cli_obj.json:
+        click.echo(json.dumps(albums, ensure_ascii=False))
+    else:
+        click.echo(yaml.dump(albums, sort_keys=False, allow_unicode=True))

osxphotos/cli/cli.py

@@ -0,0 +1,85 @@
+"""Command line interface for osxphotos """
+
+import click
+
+import osxphotos
+from osxphotos._version import __version__
+
+from .about import about
+from .albums import albums
+from .common import DB_OPTION, JSON_OPTION, OSXPHOTOS_HIDDEN
+from .debug_dump import debug_dump
+from .dump import dump
+from .export import export
+from .exportdb import exportdb
+from .grep import grep
+from .help import help
+from .info import info
+from .install_uninstall_run import install, run, uninstall
+from .keywords import keywords
+from .labels import labels
+from .list import _list_libraries, list_libraries
+from .persons import persons
+from .places import places
+from .query import query
+from .repl import repl
+from .snap_diff import diff, snap
+from .tutorial import tutorial
+from .uuid import uuid
+
+
+# Click CLI object & context settings
+class CLI_Obj:
+    def __init__(self, db=None, json=False, debug=False, group=None):
+        if debug:
+            osxphotos._set_debug(True)
+        self.db = db
+        self.json = json
+        self.group = group
+
+
+CTX_SETTINGS = dict(help_option_names=["-h", "--help"])
+
+
+@click.group(context_settings=CTX_SETTINGS)
+@DB_OPTION
+@JSON_OPTION
+@click.option(
+    "--debug",
+    required=False,
+    is_flag=True,
+    help="Enable debug output",
+    hidden=OSXPHOTOS_HIDDEN,
+)
+@click.version_option(__version__, "--version", "-v")
+@click.pass_context
+def cli_main(ctx, db, json_, debug):
+    ctx.obj = CLI_Obj(db=db, json=json_, group=cli_main)
+
+
+# install CLI commands
+for command in [
+    about,
+    albums,
+    debug_dump,
+    diff,
+    dump,
+    export,
+    exportdb,
+    grep,
+    help,
+    info,
+    install,
+    keywords,
+    labels,
+    list_libraries,
+    persons,
+    places,
+    query,
+    repl,
+    snap,
+    tutorial,
+    uninstall,
+    uuid,
+]:
+    cli_main.add_command(command)

osxphotos/cli/common.py

@@ -0,0 +1,538 @@
+"""Globals and constants use by the CLI commands"""
+
+import datetime
+import os
+import pathlib
+from typing import Callable
+
+import click
+
+import osxphotos
+from osxphotos._version import __version__
+
+from .param_types import *
+
+from rich import print as rprint
+
+# global variable to control debug output
+# set via --debug
+DEBUG = False
+
+# used to show/hide hidden commands
+OSXPHOTOS_HIDDEN = not bool(os.getenv("OSXPHOTOS_SHOW_HIDDEN", default=False))
+
+# used by snap and diff commands
+OSXPHOTOS_SNAPSHOT_DIR = "/private/tmp/osxphotos_snapshots"
+
+# where to write the crash report if osxphotos crashes
+OSXPHOTOS_CRASH_LOG = os.getcwd() + "/osxphotos_crash.log"
+
+CLI_COLOR_ERROR = "red"
+CLI_COLOR_WARNING = "yellow"
+
+
+def set_debug(debug: bool):
+    """set debug flag"""
+    global DEBUG
+    DEBUG = debug
+
+
+def is_debug():
+    """return debug flag"""
+    return DEBUG
+
+
+def noop(*args, **kwargs):
+    """no-op function"""
+    pass
+
+
+def verbose_print(
+    verbose: bool = True, timestamp: bool = False, rich=False
+) -> Callable:
+    """Create verbose function to print output
+
+    Args:
+        verbose: if True, returns verbose print function otherwise returns no-op function
+        timestamp: if True, includes timestamp in verbose output
+        rich: use rich.print instead of click.echo
+
+    Returns:
+        function to print output
+    """
+    if not verbose:
+        return noop
+
+    # closure to capture timestamp
+    def verbose_(*args, **kwargs):
+        """print output if verbose flag set"""
+        styled_args = []
+        timestamp_str = str(datetime.datetime.now()) + " -- " if timestamp else ""
+        for arg in args:
+            if type(arg) == str:
+                arg = timestamp_str + arg
+                if "error" in arg.lower():
+                    arg = click.style(arg, fg=CLI_COLOR_ERROR)
+                elif "warning" in arg.lower():
+                    arg = click.style(arg, fg=CLI_COLOR_WARNING)
+            styled_args.append(arg)
+        click.echo(*styled_args, **kwargs)
+
+    def rich_verbose_(*args, **kwargs):
+        """print output if verbose flag set using rich.print"""
+        timestamp_str = str(datetime.datetime.now()) + " -- " if timestamp else ""
+        for arg in args:
+            if type(arg) == str:
+                arg = timestamp_str + arg
+                if "error" in arg.lower():
+                    arg = f"[{CLI_COLOR_ERROR}]{arg}[/{CLI_COLOR_ERROR}]"
+                elif "warning" in arg.lower():
+                    arg = f"[{CLI_COLOR_WARNING}]{arg}[/{CLI_COLOR_WARNING}]"
+            rprint(arg, **kwargs)
+
+    return rich_verbose_ if rich else verbose_
+
+
+def get_photos_db(*db_options):
+    """Return path to photos db, select first non-None db_options
+    If no db_options are non-None, try to find library to use in
+    the following order:
+    - last library opened
+    - system library
+    - ~/Pictures/Photos Library.photoslibrary
+    - failing above, returns None
+    """
+    if db_options:
+        for db in db_options:
+            if db is not None:
+                return db
+
+    # if get here, no valid database paths passed, so try to figure out which to use
+    db = osxphotos.utils.get_last_library_path()
+    if db is not None:
+        click.echo(f"Using last opened Photos library: {db}", err=True)
+        return db
+
+    db = osxphotos.utils.get_system_library_path()
+    if db is not None:
+        click.echo(f"Using system Photos library: {db}", err=True)
+        return db
+
+    db = os.path.expanduser("~/Pictures/Photos Library.photoslibrary")
+    if os.path.isdir(db):
+        click.echo(f"Using Photos library: {db}", err=True)
+        return db
+    else:
+        return None
+
+
+DB_OPTION = click.option(
+    "--db",
+    required=False,
+    metavar="<Photos database path>",
+    default=None,
+    help=(
+        "Specify Photos database path. "
+        "Path to Photos library/database can be specified using either --db "
+        "or directly as PHOTOS_LIBRARY positional argument. "
+        "If neither --db or PHOTOS_LIBRARY provided, will attempt to find the library "
+        "to use in the following order: 1. last opened library, 2. system library, 3. ~/Pictures/Photos Library.photoslibrary"
+    ),
+    type=click.Path(exists=True),
+)
+
+DB_ARGUMENT = click.argument("photos_library", nargs=-1, type=click.Path(exists=True))
+
+JSON_OPTION = click.option(
+    "--json",
+    "json_",
+    required=False,
+    is_flag=True,
+    default=False,
+    help="Print output in JSON format.",
+)
+
+
+def DELETED_OPTIONS(f):
+    o = click.option
+    options = [
+        o(
+            "--deleted",
+            is_flag=True,
+            help="Include photos from the 'Recently Deleted' folder.",
+        ),
+        o(
+            "--deleted-only",
+            is_flag=True,
+            help="Include only photos from the 'Recently Deleted' folder.",
+        ),
+    ]
+    for o in options[::-1]:
+        f = o(f)
+    return f
+
+
+def QUERY_OPTIONS(f):
+    o = click.option
+    options = [
+        o(
+            "--keyword",
+            metavar="KEYWORD",
+            default=None,
+            multiple=True,
+            help="Search for photos with keyword KEYWORD. "
+            'If more than one keyword, treated as "OR", e.g. find photos matching any keyword',
+        ),
+        o(
+            "--person",
+            metavar="PERSON",
+            default=None,
+            multiple=True,
+            help="Search for photos with person PERSON. "
+            'If more than one person, treated as "OR", e.g. find photos matching any person',
+        ),
+        o(
+            "--album",
+            metavar="ALBUM",
+            default=None,
+            multiple=True,
+            help="Search for photos in album ALBUM. "
+            'If more than one album, treated as "OR", e.g. find photos matching any album',
+        ),
+        o(
+            "--folder",
+            metavar="FOLDER",
+            default=None,
+            multiple=True,
+            help="Search for photos in an album in folder FOLDER. "
+            'If more than one folder, treated as "OR", e.g. find photos in any FOLDER.  '
+            "Only searches top level folders (e.g. does not look at subfolders)",
+        ),
+        o(
+            "--name",
+            metavar="FILENAME",
+            default=None,
+            multiple=True,
+            help="Search for photos with filename matching FILENAME. "
+            'If more than one --name options is specified, they are treated as "OR", '
+            "e.g. find photos matching any FILENAME. ",
+        ),
+        o(
+            "--uuid",
+            metavar="UUID",
+            default=None,
+            multiple=True,
+            help="Search for photos with UUID(s). "
+            "May be repeated to include multiple UUIDs.",
+        ),
+        o(
+            "--uuid-from-file",
+            metavar="FILE",
+            default=None,
+            multiple=False,
+            help="Search for photos with UUID(s) loaded from FILE. "
+            "Format is a single UUID per line.  Lines preceded with # are ignored.",
+            type=click.Path(exists=True),
+        ),
+        o(
+            "--title",
+            metavar="TITLE",
+            default=None,
+            multiple=True,
+            help="Search for TITLE in title of photo.",
+        ),
+        o("--no-title", is_flag=True, help="Search for photos with no title."),
+        o(
+            "--description",
+            metavar="DESC",
+            default=None,
+            multiple=True,
+            help="Search for DESC in description of photo.",
+        ),
+        o(
+            "--no-description",
+            is_flag=True,
+            help="Search for photos with no description.",
+        ),
+        o(
+            "--place",
+            metavar="PLACE",
+            default=None,
+            multiple=True,
+            help="Search for PLACE in photo's reverse geolocation info",
+        ),
+        o(
+            "--no-place",
+            is_flag=True,
+            help="Search for photos with no associated place name info (no reverse geolocation info)",
+        ),
+        o(
+            "--location",
+            is_flag=True,
+            help="Search for photos with associated location info (e.g. GPS coordinates)",
+        ),
+        o(
+            "--no-location",
+            is_flag=True,
+            help="Search for photos with no associated location info (e.g. no GPS coordinates)",
+        ),
+        o(
+            "--label",
+            metavar="LABEL",
+            multiple=True,
+            help="Search for photos with image classification label LABEL (Photos 5 only). "
+            'If more than one label, treated as "OR", e.g. find photos matching any label',
+        ),
+        o(
+            "--uti",
+            metavar="UTI",
+            default=None,
+            multiple=False,
+            help="Search for photos whose uniform type identifier (UTI) matches UTI",
+        ),
+        o(
+            "-i",
+            "--ignore-case",
+            is_flag=True,
+            help="Case insensitive search for title, description, place, keyword, person, or album.",
+        ),
+        o("--edited", is_flag=True, help="Search for photos that have been edited."),
+        o(
+            "--external-edit",
+            is_flag=True,
+            help="Search for photos edited in external editor.",
+        ),
+        o("--favorite", is_flag=True, help="Search for photos marked favorite."),
+        o(
+            "--not-favorite",
+            is_flag=True,
+            help="Search for photos not marked favorite.",
+        ),
+        o("--hidden", is_flag=True, help="Search for photos marked hidden."),
+        o("--not-hidden", is_flag=True, help="Search for photos not marked hidden."),
+        o(
+            "--shared",
+            is_flag=True,
+            help="Search for photos in shared iCloud album (Photos 5 only).",
+        ),
+        o(
+            "--not-shared",
+            is_flag=True,
+            help="Search for photos not in shared iCloud album (Photos 5 only).",
+        ),
+        o(
+            "--burst",
+            is_flag=True,
+            help="Search for photos that were taken in a burst.",
+        ),
+        o(
+            "--not-burst",
+            is_flag=True,
+            help="Search for photos that are not part of a burst.",
+        ),
+        o("--live", is_flag=True, help="Search for Apple live photos"),
+        o(
+            "--not-live",
+            is_flag=True,
+            help="Search for photos that are not Apple live photos.",
+        ),
+        o("--portrait", is_flag=True, help="Search for Apple portrait mode photos."),
+        o(
+            "--not-portrait",
+            is_flag=True,
+            help="Search for photos that are not Apple portrait mode photos.",
+        ),
+        o("--screenshot", is_flag=True, help="Search for screenshot photos."),
+        o(
+            "--not-screenshot",
+            is_flag=True,
+            help="Search for photos that are not screenshot photos.",
+        ),
+        o("--slow-mo", is_flag=True, help="Search for slow motion videos."),
+        o(
+            "--not-slow-mo",
+            is_flag=True,
+            help="Search for photos that are not slow motion videos.",
+        ),
+        o("--time-lapse", is_flag=True, help="Search for time lapse videos."),
+        o(
+            "--not-time-lapse",
+            is_flag=True,
+            help="Search for photos that are not time lapse videos.",
+        ),
+        o("--hdr", is_flag=True, help="Search for high dynamic range (HDR) photos."),
+        o("--not-hdr", is_flag=True, help="Search for photos that are not HDR photos."),
+        o(
+            "--selfie",
+            is_flag=True,
+            help="Search for selfies (photos taken with front-facing cameras).",
+        ),
+        o("--not-selfie", is_flag=True, help="Search for photos that are not selfies."),
+        o("--panorama", is_flag=True, help="Search for panorama photos."),
+        o(
+            "--not-panorama",
+            is_flag=True,
+            help="Search for photos that are not panoramas.",
+        ),
+        o(
+            "--has-raw",
+            is_flag=True,
+            help="Search for photos with both a jpeg and raw version",
+        ),
+        o(
+            "--only-movies",
+            is_flag=True,
+            help="Search only for movies (default searches both images and movies).",
+        ),
+        o(
+            "--only-photos",
+            is_flag=True,
+            help="Search only for photos/images (default searches both images and movies).",
+        ),
+        o(
+            "--from-date",
+            help="Search by item start date, e.g. 2000-01-12T12:00:00, 2001-01-12T12:00:00-07:00, or 2000-12-31 (ISO 8601 with/without timezone).",
+            type=DateTimeISO8601(),
+        ),
+        o(
+            "--to-date",
+            help="Search by item end date, e.g. 2000-01-12T12:00:00, 2001-01-12T12:00:00-07:00, or 2000-12-31 (ISO 8601 with/without timezone).",
+            type=DateTimeISO8601(),
+        ),
+        o(
+            "--from-time",
+            help="Search by item start time of day, e.g. 12:00, or 12:00:00.",
+            type=TimeISO8601(),
+        ),
+        o(
+            "--to-time",
+            help="Search by item end time of day, e.g. 12:00 or 12:00:00.",
+            type=TimeISO8601(),
+        ),
+        o("--has-comment", is_flag=True, help="Search for photos that have comments."),
+        o("--no-comment", is_flag=True, help="Search for photos with no comments."),
+        o("--has-likes", is_flag=True, help="Search for photos that have likes."),
+        o("--no-likes", is_flag=True, help="Search for photos with no likes."),
+        o(
+            "--is-reference",
+            is_flag=True,
+            help="Search for photos that were imported as referenced files (not copied into Photos library).",
+        ),
+        o(
+            "--in-album",
+            is_flag=True,
+            help="Search for photos that are in one or more albums.",
+        ),
+        o(
+            "--not-in-album",
+            is_flag=True,
+            help="Search for photos that are not in any albums.",
+        ),
+        o(
+            "--duplicate",
+            is_flag=True,
+            help="Search for photos with possible duplicates. osxphotos will compare signatures of photos, "
+            "evaluating date created, size, height, width, and edited status to find *possible* duplicates. "
+            "This does not compare images byte-for-byte nor compare hashes but should find photos imported multiple "
+            "times or duplicated within Photos.",
+        ),
+        o(
+            "--min-size",
+            metavar="SIZE",
+            type=BitMathSize(),
+            help="Search for photos with size >= SIZE bytes. "
+            "The size evaluated is the photo's original size (when imported to Photos). "
+            "Size may be specified as integer bytes or using SI or NIST units. "
+            "For example, the following are all valid and equivalent sizes: '1048576' '1.048576MB', '1 MiB'.",
+        ),
+        o(
+            "--max-size",
+            metavar="SIZE",
+            type=BitMathSize(),
+            help="Search for photos with size <= SIZE bytes. "
+            "The size evaluated is the photo's original size (when imported to Photos). "
+            "Size may be specified as integer bytes or using SI or NIST units. "
+            "For example, the following are all valid and equivalent sizes: '1048576' '1.048576MB', '1 MiB'.",
+        ),
+        o(
+            "--regex",
+            metavar="REGEX TEMPLATE",
+            nargs=2,
+            multiple=True,
+            help="Search for photos where TEMPLATE matches regular expression REGEX. "
+            "For example, to find photos in an album that begins with 'Beach': '--regex \"^Beach\" \"{album}\"'. "
+            "You may specify more than one regular expression match by repeating '--regex' with different arguments.",
+        ),
+        o(
+            "--selected",
+            is_flag=True,
+            help="Filter for photos that are currently selected in Photos.",
+        ),
+        o(
+            "--exif",
+            metavar="EXIF_TAG VALUE",
+            nargs=2,
+            multiple=True,
+            help="Search for photos where EXIF_TAG exists in photo's EXIF data and contains VALUE. "
+            "For example, to find photos created by Adobe Photoshop: `--exif Software 'Adobe Photoshop' `"
+            "or to find all photos shot on a Canon camera: `--exif Make Canon`. "
+            "EXIF_TAG can be any valid exiftool tag, with or without group name, e.g. `EXIF:Make` or `Make`. "
+            "To use --exif, exiftool must be installed and in the path.",
+        ),
+        o(
+            "--query-eval",
+            metavar="CRITERIA",
+            multiple=True,
+            help="Evaluate CRITERIA to filter photos. "
+            "CRITERIA will be evaluated in context of the following python list comprehension: "
+            "`photos = [photo for photo in photos if CRITERIA]` "
+            "where photo represents a PhotoInfo object. "
+            "For example: `--query-eval photo.favorite` returns all photos that have been "
+            "favorited and is equivalent to --favorite. "
+            "You may specify more than one CRITERIA by using --query-eval multiple times. "
+            "CRITERIA must be a valid python expression. "
+            "See https://rhettbull.github.io/osxphotos/ for additional documentation on the PhotoInfo class.",
+        ),
+        o(
+            "--query-function",
+            metavar="filename.py::function",
+            multiple=True,
+            type=FunctionCall(),
+            help="Run function to filter photos. Use this in format: --query-function filename.py::function where filename.py is a python "
+            + "file you've created and function is the name of the function in the python file you want to call. "
+            + "Your function will be passed a list of PhotoInfo objects and is expected to return a filtered list of PhotoInfo objects. "
+            + "You may use more than one function by repeating the --query-function option with a different value. "
+            + "Your query function will be called after all other query options have been evaluated. "
+            + "See https://github.com/RhetTbull/osxphotos/blob/master/examples/query_function.py for example of how to use this option.",
+        ),
+    ]
+    for o in options[::-1]:
+        f = o(f)
+    return f
+
+
+def load_uuid_from_file(filename):
+    """Load UUIDs from file.  Does not validate UUIDs.
+        Format is 1 UUID per line, any line beginning with # is ignored.
+        Whitespace is stripped.
+
+    Arguments:
+        filename: file name of the file containing UUIDs
+
+    Returns:
+        list of UUIDs or empty list of no UUIDs in file
+
+    Raises:
+        FileNotFoundError if file does not exist
+    """
+
+    if not pathlib.Path(filename).is_file():
+        raise FileNotFoundError(f"Could not find file {filename}")
+
+    uuid = []
+    with open(filename, "r") as uuid_file:
+        for line in uuid_file:
+            line = line.strip()
+            if len(line) and line[0] != "#":
+                uuid.append(line)
+    return uuid

osxphotos/cli/debug_dump.py

@@ -0,0 +1,103 @@
+"""debug-dump command for osxphotos CLI"""
+
+import pprint
+import time
+
+import click
+from rich import print
+
+import osxphotos
+from osxphotos._constants import _PHOTOS_4_VERSION, _UNKNOWN_PLACE
+
+from .common import (
+    DB_ARGUMENT,
+    DB_OPTION,
+    JSON_OPTION,
+    OSXPHOTOS_HIDDEN,
+    get_photos_db,
+    verbose_print,
+)
+from .list import _list_libraries
+
+
+@click.command(hidden=OSXPHOTOS_HIDDEN)
+@DB_OPTION
+@DB_ARGUMENT
+@click.option(
+    "--dump",
+    metavar="ATTR",
+    help="Name of PhotosDB attribute to print; "
+    + "can also use albums, persons, keywords, photos to dump related attributes.",
+    multiple=True,
+)
+@click.option(
+    "--uuid",
+    metavar="UUID",
+    help="Use with '--dump photos' to dump only certain UUIDs. "
+    "May be repeated to include multiple UUIDs.",
+    multiple=True,
+)
+@click.option("--verbose", "-V", "verbose", is_flag=True, help="Print verbose output.")
+@click.pass_obj
+@click.pass_context
+def debug_dump(ctx, cli_obj, db, photos_library, dump, uuid, verbose):
+    """Print out debug info"""
+
+    verbose_ = verbose_print(verbose, rich=True)
+    db = get_photos_db(*photos_library, db, cli_obj.db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["debug-dump"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    start_t = time.perf_counter()
+    print(f"Opening database: {db}")
+    photosdb = osxphotos.PhotosDB(dbfile=db, verbose=verbose_)
+    stop_t = time.perf_counter()
+    print(f"Done; took {(stop_t-start_t):.2f} seconds")
+
+    for attr in dump:
+        if attr == "albums":
+            print("_dbalbums_album:")
+            pprint.pprint(photosdb._dbalbums_album)
+            print("_dbalbums_uuid:")
+            pprint.pprint(photosdb._dbalbums_uuid)
+            print("_dbalbum_details:")
+            pprint.pprint(photosdb._dbalbum_details)
+            print("_dbalbum_folders:")
+            pprint.pprint(photosdb._dbalbum_folders)
+            print("_dbfolder_details:")
+            pprint.pprint(photosdb._dbfolder_details)
+        elif attr == "keywords":
+            print("_dbkeywords_keyword:")
+            pprint.pprint(photosdb._dbkeywords_keyword)
+            print("_dbkeywords_uuid:")
+            pprint.pprint(photosdb._dbkeywords_uuid)
+        elif attr == "persons":
+            print("_dbfaces_uuid:")
+            pprint.pprint(photosdb._dbfaces_uuid)
+            print("_dbfaces_pk:")
+            pprint.pprint(photosdb._dbfaces_pk)
+            print("_dbpersons_pk:")
+            pprint.pprint(photosdb._dbpersons_pk)
+            print("_dbpersons_fullname:")
+            pprint.pprint(photosdb._dbpersons_fullname)
+        elif attr == "photos":
+            if uuid:
+                for uuid_ in uuid:
+                    print(f"_dbphotos['{uuid_}']:")
+                    try:
+                        pprint.pprint(photosdb._dbphotos[uuid_])
+                    except KeyError:
+                        print(f"Did not find uuid {uuid_} in _dbphotos")
+            else:
+                print("_dbphotos:")
+                pprint.pprint(photosdb._dbphotos)
+        else:
+            try:
+                val = getattr(photosdb, attr)
+                print(f"{attr}:")
+                pprint.pprint(val)
+            except Exception:
+                print(f"Did not find attribute {attr} in PhotosDB")

osxphotos/cli/dump.py

@@ -0,0 +1,44 @@
+"""dump command for osxphotos CLI """
+
+import click
+
+import osxphotos
+from osxphotos.queryoptions import QueryOptions
+
+from .common import DB_ARGUMENT, DB_OPTION, DELETED_OPTIONS, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+from .print_photo_info import print_photo_info
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DELETED_OPTIONS
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def dump(ctx, cli_obj, db, json_, deleted, deleted_only, photos_library):
+    """Print list of all photos & associated info from the Photos library."""
+
+    db = get_photos_db(*photos_library, db, cli_obj.db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["dump"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    # check exclusive options
+    if deleted and deleted_only:
+        click.echo("Incompatible dump options", err=True)
+        click.echo(ctx.obj.group.commands["dump"].get_help(ctx), err=True)
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    if deleted or deleted_only:
+        photos = photosdb.photos(movies=True, intrash=True)
+    else:
+        photos = []
+    if not deleted_only:
+        photos += photosdb.photos(movies=True)
+
+    print_photo_info(photos, json_ or cli_obj.json)

osxphotos/cli/exportdb.py

@@ -0,0 +1,251 @@
+"""exportdb command for osxphotos CLI"""
+
+import pathlib
+import sys
+
+import click
+from rich import print
+
+from osxphotos._constants import OSXPHOTOS_EXPORT_DB
+from osxphotos._version import __version__
+from osxphotos.export_db import OSXPHOTOS_EXPORTDB_VERSION, ExportDB
+from osxphotos.export_db_utils import (
+    export_db_check_signatures,
+    export_db_get_last_run,
+    export_db_get_version,
+    export_db_save_config_to_file,
+    export_db_touch_files,
+    export_db_update_signatures,
+    export_db_vacuum,
+)
+
+from .common import OSXPHOTOS_HIDDEN, verbose_print
+
+
+@click.command(name="exportdb", hidden=OSXPHOTOS_HIDDEN)
+@click.option("--version", is_flag=True, help="Print export database version and exit.")
+@click.option("--vacuum", is_flag=True, help="Run VACUUM to defragment the database.")
+@click.option(
+    "--check-signatures",
+    is_flag=True,
+    help="Check signatures for all exported photos in the database to find signatures that don't match.",
+)
+@click.option(
+    "--update-signatures",
+    is_flag=True,
+    help="Update signatures for all exported photos in the database to match on-disk signatures.",
+)
+@click.option(
+    "--touch-file",
+    is_flag=True,
+    help="Touch files on disk to match created date in Photos library and update export database signatures",
+)
+@click.option(
+    "--last-run",
+    is_flag=True,
+    help="Show last run osxphotos commands used with this database.",
+)
+@click.option(
+    "--save-config",
+    metavar="CONFIG_FILE",
+    help="Save last run configuration to TOML file for use by --load-config.",
+)
+@click.option(
+    "--info",
+    metavar="FILE_PATH",
+    nargs=1,
+    help="Print information about FILE_PATH contained in the database.",
+)
+@click.option(
+    "--migrate",
+    is_flag=True,
+    help="Migrate (if needed) export database to current version.",
+)
+@click.option(
+    "--sql",
+    metavar="SQL_STATEMENT",
+    help="Execute SQL_STATEMENT against export database and print results.",
+)
+@click.option(
+    "--export-dir",
+    help="Optional path to export directory (if not parent of export database).",
+    type=click.Path(exists=True, file_okay=False, dir_okay=True),
+)
+@click.option("--verbose", "-V", is_flag=True, help="Print verbose output.")
+@click.option(
+    "--dry-run",
+    is_flag=True,
+    help="Run in dry-run mode (don't actually update files), e.g. for use with --update-signatures.",
+)
+@click.argument("export_db", metavar="EXPORT_DATABASE", type=click.Path(exists=True))
+def exportdb(
+    version,
+    vacuum,
+    check_signatures,
+    update_signatures,
+    touch_file,
+    last_run,
+    save_config,
+    info,
+    migrate,
+    sql,
+    export_dir,
+    verbose,
+    dry_run,
+    export_db,
+):
+    """Utilities for working with the osxphotos export database"""
+
+    verbose_ = verbose_print(verbose, rich=True)
+
+    export_db = pathlib.Path(export_db)
+    if export_db.is_dir():
+        # assume it's the export folder
+        export_db = export_db / OSXPHOTOS_EXPORT_DB
+        if not export_db.is_file():
+            print(
+                f"[red]Error: {OSXPHOTOS_EXPORT_DB} missing from {export_db.parent}[/red]"
+            )
+            sys.exit(1)
+
+    export_dir = export_dir or export_db.parent
+
+    sub_commands = [
+        version,
+        check_signatures,
+        update_signatures,
+        touch_file,
+        last_run,
+        bool(save_config),
+        bool(info),
+        migrate,
+        bool(sql),
+    ]
+    if sum(sub_commands) > 1:
+        print("[red]Only a single sub-command may be specified at a time[/red]")
+        sys.exit(1)
+
+    if version:
+        try:
+            osxphotos_ver, export_db_ver = export_db_get_version(export_db)
+        except Exception as e:
+            print(f"[red]Error: could not read version from {export_db}: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(
+                f"osxphotos version: {osxphotos_ver}, export database version: {export_db_ver}"
+            )
+        sys.exit(0)
+
+    if vacuum:
+        try:
+            start_size = pathlib.Path(export_db).stat().st_size
+            export_db_vacuum(export_db)
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(
+                f"Vacuumed {export_db}! {start_size} bytes -> {pathlib.Path(export_db).stat().st_size} bytes"
+            )
+            sys.exit(0)
+
+    if update_signatures:
+        try:
+            updated, skipped = export_db_update_signatures(
+                export_db, export_dir, verbose_, dry_run
+            )
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(f"Done. Updated {updated} files, skipped {skipped} files.")
+            sys.exit(0)
+
+    if last_run:
+        try:
+            last_run_info = export_db_get_last_run(export_db)
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(f"last run at {last_run_info[0]}:")
+            print(f"osxphotos {last_run_info[1]}")
+            sys.exit(0)
+
+    if save_config:
+        try:
+            export_db_save_config_to_file(export_db, save_config)
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(f"Saved configuration to {save_config}")
+            sys.exit(0)
+
+    if check_signatures:
+        try:
+            matched, notmatched, skipped = export_db_check_signatures(
+                export_db, export_dir, verbose_=verbose_
+            )
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(
+                f"Done. Found {matched} matching signatures and {notmatched} signatures that don't match. Skipped {skipped} missing files."
+            )
+            sys.exit(0)
+
+    if touch_file:
+        try:
+            touched, not_touched, skipped = export_db_touch_files(
+                export_db, export_dir, verbose_=verbose_, dry_run=dry_run
+            )
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            print(
+                f"Done. Touched {touched} files, skipped {not_touched} up to date files, skipped {skipped} missing files."
+            )
+            sys.exit(0)
+
+    if info:
+        exportdb = ExportDB(export_db, export_dir)
+        try:
+            info_rec = exportdb.get_file_record(info)
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            if info_rec:
+                print(info_rec.asdict())
+            else:
+                print(f"[red]File '{info}' not found in export database[/red]")
+            sys.exit(0)
+
+    if migrate:
+        exportdb = ExportDB(export_db, export_dir)
+        if upgraded := exportdb.was_upgraded:
+            print(
+                f"Migrated export database {export_db} from version {upgraded[0]} to {upgraded[1]}"
+            )
+        else:
+            print(
+                f"Export database {export_db} is already at latest version {OSXPHOTOS_EXPORTDB_VERSION}"
+            )
+        sys.exit(0)
+
+    if sql:
+        exportdb = ExportDB(export_db, export_dir)
+        try:
+            c = exportdb._conn.cursor()
+            results = c.execute(sql)
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            for row in results:
+                print(row)
+            sys.exit(0)

osxphotos/cli/grep.py

@@ -0,0 +1,57 @@
+"""grep command for osxphotos CLI """
+
+import pathlib
+
+import click
+from rich import print
+
+from osxphotos.photosdb.photosdb_utils import get_photos_library_version
+from osxphotos.sqlgrep import sqlgrep
+
+from .common import DB_OPTION, OSXPHOTOS_HIDDEN, get_photos_db
+
+
+@click.command(name="grep", hidden=OSXPHOTOS_HIDDEN)
+@DB_OPTION
+@click.pass_obj
+@click.pass_context
+@click.option(
+    "--ignore-case",
+    "-i",
+    required=False,
+    is_flag=True,
+    default=False,
+    help="Ignore case when searching (default is case-sensitive).",
+)
+@click.option(
+    "--print-filename",
+    "-p",
+    required=False,
+    is_flag=True,
+    default=False,
+    help="Print name of database file when printing results.",
+)
+@click.argument("pattern", metavar="PATTERN", required=True)
+def grep(ctx, cli_obj, db, ignore_case, print_filename, pattern):
+    """Search for PATTERN in the Photos sqlite database file"""
+    db = db or get_photos_db()
+    db = pathlib.Path(db)
+    if db.is_file():
+        # if passed the actual database, really want the parent of the database directory
+        db = db.parent.parent
+    photos_ver = get_photos_library_version(str(db))
+    if photos_ver < 5:
+        db_file = db / "database" / "photos.db"
+    else:
+        db_file = db / "database" / "Photos.sqlite"
+
+    if not db_file.is_file():
+        click.secho(f"Could not find database file {db_file}", fg="red")
+        ctx.exit(2)
+
+    db_file = str(db_file)
+
+    for table, column, row_id, value in sqlgrep(
+        db_file, pattern, ignore_case, print_filename, rich_markup=True
+    ):
+        print(", ".join([table, column, row_id, value]))

osxphotos/cli/help.py

@@ -1,7 +1,6 @@
 """Help text helper class for osxphotos CLI """
 
 import io
-import pathlib
 import re
 
 import click
@@ -9,13 +8,13 @@ import osxmetadata
 from rich.console import Console
 from rich.markdown import Markdown
 
-from ._constants import (
+from osxphotos._constants import (
     EXTENDED_ATTRIBUTE_NAMES,
     EXTENDED_ATTRIBUTE_NAMES_QUOTED,
     OSXPHOTOS_EXPORT_DB,
     POST_COMMAND_CATEGORIES,
 )
-from .phototemplate import (
+from osxphotos.phototemplate import (
     TEMPLATE_SUBSTITUTIONS,
     TEMPLATE_SUBSTITUTIONS_MULTI_VALUED,
     TEMPLATE_SUBSTITUTIONS_PATHLIB,
@@ -25,15 +24,37 @@ from .phototemplate import (
 __all__ = [
     "ExportCommand",
     "template_help",
-    "tutorial_help",
     "rich_text",
     "strip_md_header_and_links",
     "strip_md_links",
     "strip_html_comments",
-    "get_tutorial_text",
+    "help",
+    "get_help_msg",
 ]
 
 
+def get_help_msg(command):
+    """get help message for a Click command"""
+    with click.Context(command) as ctx:
+        return command.get_help(ctx)
+
+
+@click.command()
+@click.argument("topic", default=None, required=False, nargs=1)
+@click.pass_context
+def help(ctx, topic, **kw):
+    """Print help; for help on commands: help <command>."""
+    if topic is None:
+        click.echo(ctx.parent.get_help())
+        return
+    elif topic in ctx.obj.group.commands:
+        ctx.info_name = topic
+        click.echo_via_pager(ctx.obj.group.commands[topic].get_help(ctx))
+    else:
+        click.echo(f"Invalid command: {topic}", err=True)
+        click.echo(ctx.parent.get_help())
+
+
 # TODO: The following help text could probably be done as mako template
 class ExportCommand(click.Command):
     """Custom click.Command that overrides get_help() to show additional help info for export"""
@@ -282,19 +303,6 @@ def template_help(width=78):
     return help_str
 
 
-def tutorial_help(width=78):
-    """Return formatted string for tutorial"""
-    sio = io.StringIO()
-    console = Console(file=sio, force_terminal=True, width=width)
-    help_md = get_tutorial_text()
-    help_md = strip_html_comments(help_md)
-    help_md = strip_md_links(help_md)
-    console.print(Markdown(help_md))
-    help_str = sio.getvalue()
-    sio.close()
-    return help_str
-
-
 def rich_text(text, width=78):
     """Return rich formatted text"""
     sio = io.StringIO()
@@ -348,12 +356,3 @@ def strip_md_links(md):
 def strip_html_comments(text):
     """Strip html comments from text (which doesn't need to be valid HTML)"""
     return re.sub(r"<!--(.|\s|\n)*?-->", "", text)
-
-
-def get_tutorial_text():
-    """Load tutorial text from file"""
-    # TODO: would be better to use importlib.abc.ResourceReader but I can't find a single example of how to do this
-    help_file = pathlib.Path(__file__).parent / "tutorial.md"
-    with open(help_file, "r") as fd:
-        md = fd.read()
-    return md

osxphotos/cli/info.py

@@ -0,0 +1,72 @@
+"""info command for osxphotos CLI"""
+
+import json
+
+import click
+import yaml
+
+import osxphotos
+from osxphotos._constants import _PHOTOS_4_VERSION
+
+from .common import DB_ARGUMENT, DB_OPTION, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def info(ctx, cli_obj, db, json_, photos_library):
+    """Print out descriptive info of the Photos library database."""
+
+    db = get_photos_db(*photos_library, db, cli_obj.db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["info"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    info = {"database_path": photosdb.db_path, "database_version": photosdb.db_version}
+    photos = photosdb.photos(movies=False)
+    not_shared_photos = [p for p in photos if not p.shared]
+    info["photo_count"] = len(not_shared_photos)
+
+    hidden = [p for p in photos if p.hidden]
+    info["hidden_photo_count"] = len(hidden)
+
+    movies = photosdb.photos(images=False, movies=True)
+    not_shared_movies = [p for p in movies if not p.shared]
+    info["movie_count"] = len(not_shared_movies)
+
+    if photosdb.db_version > _PHOTOS_4_VERSION:
+        shared_photos = [p for p in photos if p.shared]
+        info["shared_photo_count"] = len(shared_photos)
+
+        shared_movies = [p for p in movies if p.shared]
+        info["shared_movie_count"] = len(shared_movies)
+
+    keywords = photosdb.keywords_as_dict
+    info["keywords_count"] = len(keywords)
+    info["keywords"] = keywords
+
+    albums = photosdb.albums_as_dict
+    info["albums_count"] = len(albums)
+    info["albums"] = albums
+
+    if photosdb.db_version > _PHOTOS_4_VERSION:
+        albums_shared = photosdb.albums_shared_as_dict
+        info["shared_albums_count"] = len(albums_shared)
+        info["shared_albums"] = albums_shared
+
+    persons = photosdb.persons_as_dict
+
+    info["persons_count"] = len(persons)
+    info["persons"] = persons
+
+    if cli_obj.json or json_:
+        click.echo(json.dumps(info, ensure_ascii=False))
+    else:
+        click.echo(yaml.dump(info, sort_keys=False, allow_unicode=True))

osxphotos/cli/install_uninstall_run.py

@@ -0,0 +1,37 @@
+"""install/uninstall/run commands for osxphotos CLI"""
+
+import sys
+from runpy import run_module, run_path
+
+import click
+
+
+@click.command()
+@click.argument("packages", nargs=-1, required=True)
+@click.option(
+    "-U", "--upgrade", is_flag=True, help="Upgrade packages to latest version"
+)
+def install(packages, upgrade):
+    """Install Python packages into the same environment as osxphotos"""
+    args = ["pip", "install"]
+    if upgrade:
+        args += ["--upgrade"]
+    args += list(packages)
+    sys.argv = args
+    run_module("pip", run_name="__main__")
+
+
+@click.command()
+@click.argument("packages", nargs=-1, required=True)
+@click.option("-y", "--yes", is_flag=True, help="Don't ask for confirmation")
+def uninstall(packages, yes):
+    """Uninstall Python packages from the osxphotos environment"""
+    sys.argv = ["pip", "uninstall"] + list(packages) + (["-y"] if yes else [])
+    run_module("pip", run_name="__main__")
+
+
+@click.command(name="run")
+@click.argument("python_file", nargs=1, type=click.Path(exists=True))
+def run(python_file):
+    """Run a python file using same environment as osxphotos"""
+    run_path(python_file, run_name="__main__")

osxphotos/cli/keywords.py

@@ -0,0 +1,37 @@
+"""keywords command for osxphotos CLI"""
+
+import json
+
+import click
+import yaml
+
+import osxphotos
+
+from .common import DB_ARGUMENT, DB_OPTION, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def keywords(ctx, cli_obj, db, json_, photos_library):
+    """Print out keywords found in the Photos library."""
+
+    # below needed for to make CliRunner work for testing
+    cli_db = cli_obj.db if cli_obj is not None else None
+    db = get_photos_db(*photos_library, db, cli_db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["keywords"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    keywords = {"keywords": photosdb.keywords_as_dict}
+    if json_ or cli_obj.json:
+        click.echo(json.dumps(keywords, ensure_ascii=False))
+    else:
+        click.echo(yaml.dump(keywords, sort_keys=False, allow_unicode=True))

osxphotos/cli/labels.py

@@ -0,0 +1,37 @@
+"""labels command for osxphotos CLI"""
+
+import json
+
+import click
+import yaml
+
+import osxphotos
+
+from .common import DB_ARGUMENT, DB_OPTION, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def labels(ctx, cli_obj, db, json_, photos_library):
+    """Print out image classification labels found in the Photos library."""
+
+    # below needed for to make CliRunner work for testing
+    cli_db = cli_obj.db if cli_obj is not None else None
+    db = get_photos_db(*photos_library, db, cli_db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["labels"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    labels = {"labels": photosdb.labels_as_dict}
+    if json_ or cli_obj.json:
+        click.echo(json.dumps(labels, ensure_ascii=False))
+    else:
+        click.echo(yaml.dump(labels, sort_keys=False, allow_unicode=True))

osxphotos/cli/list.py

@@ -0,0 +1,57 @@
+"""list command for osxphotos CLI"""
+
+import json
+
+import click
+
+import osxphotos
+
+from .common import JSON_OPTION
+
+
+@click.command(name="list")
+@JSON_OPTION
+@click.pass_obj
+@click.pass_context
+def list_libraries(ctx, cli_obj, json_):
+    """Print list of Photos libraries found on the system."""
+
+    # implemented in _list_libraries so it can be called by other CLI functions
+    # without errors due to passing ctx and cli_obj
+    _list_libraries(json_=json_ or cli_obj.json, error=False)
+
+
+def _list_libraries(json_=False, error=True):
+    """Print list of Photos libraries found on the system.
+    If json_ == True, print output as JSON (default = False)"""
+
+    photo_libs = osxphotos.utils.list_photo_libraries()
+    sys_lib = osxphotos.utils.get_system_library_path()
+    last_lib = osxphotos.utils.get_last_library_path()
+
+    if json_:
+        libs = {
+            "photo_libraries": photo_libs,
+            "system_library": sys_lib,
+            "last_library": last_lib,
+        }
+        click.echo(json.dumps(libs, ensure_ascii=False))
+    else:
+        last_lib_flag = sys_lib_flag = False
+
+        for lib in photo_libs:
+            if lib == sys_lib:
+                click.echo(f"(*)\t{lib}", err=error)
+                sys_lib_flag = True
+            elif lib == last_lib:
+                click.echo(f"(#)\t{lib}", err=error)
+                last_lib_flag = True
+            else:
+                click.echo(f"\t{lib}", err=error)
+
+        if sys_lib_flag or last_lib_flag:
+            click.echo("\n", err=error)
+        if sys_lib_flag:
+            click.echo("(*)\tSystem Photos Library", err=error)
+        if last_lib_flag:
+            click.echo("(#)\tLast opened Photos Library", err=error)

osxphotos/cli/param_types.py

@@ -0,0 +1,108 @@
+"""Click parameter types for osxphotos CLI"""
+import datetime
+import pathlib
+
+import bitmath
+import click
+
+from osxphotos.export_db_utils import export_db_get_version
+from osxphotos.utils import expand_and_validate_filepath, load_function
+
+__all__ = [
+    "BitMathSize",
+    "DateTimeISO8601",
+    "ExportDBType",
+    "FunctionCall",
+    "TimeISO8601",
+]
+
+
+class DateTimeISO8601(click.ParamType):
+
+    name = "DATETIME"
+
+    def convert(self, value, param, ctx):
+        try:
+            return datetime.datetime.fromisoformat(value)
+        except Exception:
+            self.fail(
+                f"Invalid datetime format {value}. "
+                "Valid format: YYYY-MM-DD[*HH[:MM[:SS[.fff[fff]]]][+HH:MM[:SS[.ffffff]]]]"
+            )
+
+
+class BitMathSize(click.ParamType):
+
+    name = "BITMATH"
+
+    def convert(self, value, param, ctx):
+        try:
+            value = bitmath.parse_string(value)
+        except ValueError:
+            # no units specified
+            try:
+                value = int(value)
+                value = bitmath.Byte(value)
+            except ValueError as e:
+                self.fail(
+                    f"{value} must be specified as bytes or using SI/NIST units. "
+                    + "For example, the following are all valid and equivalent sizes: '1048576' '1.048576MB', '1 MiB'."
+                )
+        return value
+
+
+class TimeISO8601(click.ParamType):
+
+    name = "TIME"
+
+    def convert(self, value, param, ctx):
+        try:
+            return datetime.time.fromisoformat(value).replace(tzinfo=None)
+        except Exception:
+            self.fail(
+                f"Invalid time format {value}. "
+                "Valid format: HH[:MM[:SS[.fff[fff]]]][+HH:MM[:SS[.ffffff]]] "
+                "however, note that timezone will be ignored."
+            )
+
+
+class FunctionCall(click.ParamType):
+    name = "FUNCTION"
+
+    def convert(self, value, param, ctx):
+        if "::" not in value:
+            self.fail(
+                f"Could not parse function name from '{value}'. "
+                "Valid format filename.py::function"
+            )
+
+        filename, funcname = value.split("::")
+
+        filename_validated = expand_and_validate_filepath(filename)
+        if not filename_validated:
+            self.fail(f"'{filename}' does not appear to be a file")
+
+        try:
+            function = load_function(filename_validated, funcname)
+        except Exception as e:
+            self.fail(f"Could not load function {funcname} from {filename_validated}")
+
+        return (function, value)
+
+
+class ExportDBType(click.ParamType):
+
+    name = "EXPORTDB"
+
+    def convert(self, value, param, ctx):
+        try:
+            export_db_name = pathlib.Path(value)
+            if export_db_name.is_dir():
+                raise click.BadParameter(f"{value} is a directory")
+            if export_db_name.is_file():
+                # verify it's actually an osxphotos export_db
+                # export_db_get_version will raise an error if it's not valid
+                osxphotos_ver, export_db_ver = export_db_get_version(value)
+            return value
+        except Exception:
+            self.fail(f"{value} exists but is not a valid osxphotos export database. ")

osxphotos/cli/persons.py

@@ -0,0 +1,36 @@
+"""persons command for osxphotos CLI"""
+import json
+
+import click
+import yaml
+
+import osxphotos
+
+from .common import DB_ARGUMENT, DB_OPTION, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def persons(ctx, cli_obj, db, json_, photos_library):
+    """Print out persons (faces) found in the Photos library."""
+
+    # below needed for to make CliRunner work for testing
+    cli_db = cli_obj.db if cli_obj is not None else None
+    db = get_photos_db(*photos_library, db, cli_db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["persons"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    persons = {"persons": photosdb.persons_as_dict}
+    if json_ or cli_obj.json:
+        click.echo(json.dumps(persons, ensure_ascii=False))
+    else:
+        click.echo(yaml.dump(persons, sort_keys=False, allow_unicode=True))

osxphotos/cli/places.py

@@ -0,0 +1,62 @@
+"""places command for osxphotos CLI"""
+
+import json
+
+import click
+import yaml
+
+import osxphotos
+from osxphotos._constants import _PHOTOS_4_VERSION, _UNKNOWN_PLACE
+
+from .common import DB_ARGUMENT, DB_OPTION, JSON_OPTION, get_photos_db
+from .list import _list_libraries
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def places(ctx, cli_obj, db, json_, photos_library):
+    """Print out places found in the Photos library."""
+
+    # below needed for to make CliRunner work for testing
+    cli_db = cli_obj.db if cli_obj is not None else None
+    db = get_photos_db(*photos_library, db, cli_db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["places"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    place_names = {}
+    for photo in photosdb.photos(movies=True):
+        if photo.place:
+            try:
+                place_names[photo.place.name] += 1
+            except Exception:
+                place_names[photo.place.name] = 1
+        else:
+            try:
+                place_names[_UNKNOWN_PLACE] += 1
+            except Exception:
+                place_names[_UNKNOWN_PLACE] = 1
+
+    # sort by place count
+    places = {
+        "places": {
+            name: place_names[name]
+            for name in sorted(
+                place_names.keys(), key=lambda key: place_names[key], reverse=True
+            )
+        }
+    }
+
+    # below needed for to make CliRunner work for testing
+    cli_json = cli_obj.json if cli_obj is not None else None
+    if json_ or cli_json:
+        click.echo(json.dumps(places, ensure_ascii=False))
+    else:
+        click.echo(yaml.dump(places, sort_keys=False, allow_unicode=True))

osxphotos/cli/print_photo_info.py

@@ -0,0 +1,112 @@
+"""print_photo_info function to print PhotoInfo objects"""
+
+import csv
+import sys
+from typing import Callable, List
+
+from osxphotos.photoinfo import PhotoInfo
+
+
+def print_photo_info(
+    photos: List[PhotoInfo], json: bool = False, print_func: Callable = print
+):
+    dump = []
+    if json:
+        dump.extend(p.json() for p in photos)
+        print_func(f"[{', '.join(dump)}]")
+    else:
+        # dump as CSV
+        csv_writer = csv.writer(
+            sys.stdout, delimiter=",", quotechar='"', quoting=csv.QUOTE_MINIMAL
+        )
+        # add headers
+        dump.append(
+            [
+                "uuid",
+                "filename",
+                "original_filename",
+                "date",
+                "description",
+                "title",
+                "keywords",
+                "albums",
+                "persons",
+                "path",
+                "ismissing",
+                "hasadjustments",
+                "external_edit",
+                "favorite",
+                "hidden",
+                "shared",
+                "latitude",
+                "longitude",
+                "path_edited",
+                "isphoto",
+                "ismovie",
+                "uti",
+                "burst",
+                "live_photo",
+                "path_live_photo",
+                "iscloudasset",
+                "incloud",
+                "date_modified",
+                "portrait",
+                "screenshot",
+                "slow_mo",
+                "time_lapse",
+                "hdr",
+                "selfie",
+                "panorama",
+                "has_raw",
+                "uti_raw",
+                "path_raw",
+                "intrash",
+            ]
+        )
+        for p in photos:
+            date_modified_iso = p.date_modified.isoformat() if p.date_modified else None
+            dump.append(
+                [
+                    p.uuid,
+                    p.filename,
+                    p.original_filename,
+                    p.date.isoformat(),
+                    p.description,
+                    p.title,
+                    ", ".join(p.keywords),
+                    ", ".join(p.albums),
+                    ", ".join(p.persons),
+                    p.path,
+                    p.ismissing,
+                    p.hasadjustments,
+                    p.external_edit,
+                    p.favorite,
+                    p.hidden,
+                    p.shared,
+                    p._latitude,
+                    p._longitude,
+                    p.path_edited,
+                    p.isphoto,
+                    p.ismovie,
+                    p.uti,
+                    p.burst,
+                    p.live_photo,
+                    p.path_live_photo,
+                    p.iscloudasset,
+                    p.incloud,
+                    date_modified_iso,
+                    p.portrait,
+                    p.screenshot,
+                    p.slow_mo,
+                    p.time_lapse,
+                    p.hdr,
+                    p.selfie,
+                    p.panorama,
+                    p.has_raw,
+                    p.uti_raw,
+                    p.path_raw,
+                    p.intrash,
+                ]
+            )
+        for row in dump:
+            csv_writer.writerow(row)

osxphotos/cli/query.py

@@ -0,0 +1,358 @@
+"""query command for osxphotos CLI"""
+
+import click
+
+import osxphotos
+from osxphotos.photosalbum import PhotosAlbum
+from osxphotos.queryoptions import QueryOptions
+
+from .common import (
+    CLI_COLOR_ERROR,
+    CLI_COLOR_WARNING,
+    DB_ARGUMENT,
+    DB_OPTION,
+    DELETED_OPTIONS,
+    JSON_OPTION,
+    OSXPHOTOS_HIDDEN,
+    QUERY_OPTIONS,
+    get_photos_db,
+    load_uuid_from_file,
+    set_debug,
+)
+from .list import _list_libraries
+from .print_photo_info import print_photo_info
+
+
+@click.command()
+@DB_OPTION
+@JSON_OPTION
+@QUERY_OPTIONS
+@DELETED_OPTIONS
+@click.option("--missing", is_flag=True, help="Search for photos missing from disk.")
+@click.option(
+    "--not-missing",
+    is_flag=True,
+    help="Search for photos present on disk (e.g. not missing).",
+)
+@click.option(
+    "--cloudasset",
+    is_flag=True,
+    help="Search for photos that are part of an iCloud library",
+)
+@click.option(
+    "--not-cloudasset",
+    is_flag=True,
+    help="Search for photos that are not part of an iCloud library",
+)
+@click.option(
+    "--incloud",
+    is_flag=True,
+    help="Search for photos that are in iCloud (have been synched)",
+)
+@click.option(
+    "--not-incloud",
+    is_flag=True,
+    help="Search for photos that are not in iCloud (have not been synched)",
+)
+@click.option(
+    "--add-to-album",
+    metavar="ALBUM",
+    help="Add all photos from query to album ALBUM in Photos. Album ALBUM will be created "
+    "if it doesn't exist.  All photos in the query results will be added to this album. "
+    "This only works if the Photos library being queried is the last-opened (default) library in Photos. "
+    "This feature is currently experimental.  I don't know how well it will work on large query sets.",
+)
+@click.option(
+    "--debug", required=False, is_flag=True, default=False, hidden=OSXPHOTOS_HIDDEN
+)
+@DB_ARGUMENT
+@click.pass_obj
+@click.pass_context
+def query(
+    ctx,
+    cli_obj,
+    db,
+    photos_library,
+    keyword,
+    person,
+    album,
+    folder,
+    name,
+    uuid,
+    uuid_from_file,
+    title,
+    no_title,
+    description,
+    no_description,
+    ignore_case,
+    json_,
+    edited,
+    external_edit,
+    favorite,
+    not_favorite,
+    hidden,
+    not_hidden,
+    missing,
+    not_missing,
+    shared,
+    not_shared,
+    only_movies,
+    only_photos,
+    uti,
+    burst,
+    not_burst,
+    live,
+    not_live,
+    cloudasset,
+    not_cloudasset,
+    incloud,
+    not_incloud,
+    from_date,
+    to_date,
+    from_time,
+    to_time,
+    portrait,
+    not_portrait,
+    screenshot,
+    not_screenshot,
+    slow_mo,
+    not_slow_mo,
+    time_lapse,
+    not_time_lapse,
+    hdr,
+    not_hdr,
+    selfie,
+    not_selfie,
+    panorama,
+    not_panorama,
+    has_raw,
+    place,
+    no_place,
+    location,
+    no_location,
+    label,
+    deleted,
+    deleted_only,
+    has_comment,
+    no_comment,
+    has_likes,
+    no_likes,
+    is_reference,
+    in_album,
+    not_in_album,
+    duplicate,
+    min_size,
+    max_size,
+    regex,
+    selected,
+    exif,
+    query_eval,
+    query_function,
+    add_to_album,
+    debug,
+):
+    """Query the Photos database using 1 or more search options;
+    if more than one option is provided, they are treated as "AND"
+    (e.g. search for photos matching all options).
+    """
+
+    if debug:
+        set_debug(True)
+        osxphotos._set_debug(True)
+
+    # if no query terms, show help and return
+    # sanity check input args
+    nonexclusive = [
+        keyword,
+        person,
+        album,
+        folder,
+        name,
+        uuid,
+        uuid_from_file,
+        edited,
+        external_edit,
+        uti,
+        has_raw,
+        from_date,
+        to_date,
+        from_time,
+        to_time,
+        label,
+        is_reference,
+        query_eval,
+        query_function,
+        min_size,
+        max_size,
+        regex,
+        selected,
+        exif,
+        duplicate,
+    ]
+    exclusive = [
+        (favorite, not_favorite),
+        (hidden, not_hidden),
+        (missing, not_missing),
+        (any(title), no_title),
+        (any(description), no_description),
+        (only_photos, only_movies),
+        (burst, not_burst),
+        (live, not_live),
+        (cloudasset, not_cloudasset),
+        (incloud, not_incloud),
+        (portrait, not_portrait),
+        (screenshot, not_screenshot),
+        (slow_mo, not_slow_mo),
+        (time_lapse, not_time_lapse),
+        (hdr, not_hdr),
+        (selfie, not_selfie),
+        (panorama, not_panorama),
+        (any(place), no_place),
+        (deleted, deleted_only),
+        (shared, not_shared),
+        (has_comment, no_comment),
+        (has_likes, no_likes),
+        (in_album, not_in_album),
+        (location, no_location),
+    ]
+    # print help if no non-exclusive term or a double exclusive term is given
+    if any(all(bb) for bb in exclusive) or not any(
+        nonexclusive + [b ^ n for b, n in exclusive]
+    ):
+        click.echo("Incompatible query options", err=True)
+        click.echo(ctx.obj.group.commands["query"].get_help(ctx), err=True)
+        return
+
+    # actually have something to query
+    # default searches for everything
+    photos = True
+    movies = True
+    if only_movies:
+        photos = False
+    if only_photos:
+        movies = False
+
+    # load UUIDs if necessary and append to any uuids passed with --uuid
+    if uuid_from_file:
+        uuid_list = list(uuid)  # Click option is a tuple
+        uuid_list.extend(load_uuid_from_file(uuid_from_file))
+        uuid = tuple(uuid_list)
+
+    # below needed for to make CliRunner work for testing
+    cli_db = cli_obj.db if cli_obj is not None else None
+    db = get_photos_db(*photos_library, db, cli_db)
+    if db is None:
+        click.echo(ctx.obj.group.commands["query"].get_help(ctx), err=True)
+        click.echo("\n\nLocated the following Photos library databases: ", err=True)
+        _list_libraries()
+        return
+
+    photosdb = osxphotos.PhotosDB(dbfile=db)
+    query_options = QueryOptions(
+        keyword=keyword,
+        person=person,
+        album=album,
+        folder=folder,
+        uuid=uuid,
+        title=title,
+        no_title=no_title,
+        description=description,
+        no_description=no_description,
+        ignore_case=ignore_case,
+        edited=edited,
+        external_edit=external_edit,
+        favorite=favorite,
+        not_favorite=not_favorite,
+        hidden=hidden,
+        not_hidden=not_hidden,
+        missing=missing,
+        not_missing=not_missing,
+        shared=shared,
+        not_shared=not_shared,
+        photos=photos,
+        movies=movies,
+        uti=uti,
+        burst=burst,
+        not_burst=not_burst,
+        live=live,
+        not_live=not_live,
+        cloudasset=cloudasset,
+        not_cloudasset=not_cloudasset,
+        incloud=incloud,
+        not_incloud=not_incloud,
+        from_date=from_date,
+        to_date=to_date,
+        from_time=from_time,
+        to_time=to_time,
+        portrait=portrait,
+        not_portrait=not_portrait,
+        screenshot=screenshot,
+        not_screenshot=not_screenshot,
+        slow_mo=slow_mo,
+        not_slow_mo=not_slow_mo,
+        time_lapse=time_lapse,
+        not_time_lapse=not_time_lapse,
+        hdr=hdr,
+        not_hdr=not_hdr,
+        selfie=selfie,
+        not_selfie=not_selfie,
+        panorama=panorama,
+        not_panorama=not_panorama,
+        has_raw=has_raw,
+        place=place,
+        no_place=no_place,
+        location=location,
+        no_location=no_location,
+        label=label,
+        deleted=deleted,
+        deleted_only=deleted_only,
+        has_comment=has_comment,
+        no_comment=no_comment,
+        has_likes=has_likes,
+        no_likes=no_likes,
+        is_reference=is_reference,
+        in_album=in_album,
+        not_in_album=not_in_album,
+        name=name,
+        min_size=min_size,
+        max_size=max_size,
+        query_eval=query_eval,
+        function=query_function,
+        regex=regex,
+        selected=selected,
+        exif=exif,
+        duplicate=duplicate,
+    )
+
+    try:
+        photos = photosdb.query(query_options)
+    except ValueError as e:
+        if "Invalid query_eval CRITERIA:" in str(e):
+            msg = str(e).split(":")[1]
+            raise click.BadOptionUsage(
+                "query_eval", f"Invalid query-eval CRITERIA: {msg}"
+            )
+        else:
+            raise ValueError(e)
+
+    # below needed for to make CliRunner work for testing
+    cli_json = cli_obj.json if cli_obj is not None else None
+
+    if add_to_album and photos:
+        album_query = PhotosAlbum(add_to_album, verbose=None)
+        photo_len = len(photos)
+        photo_word = "photos" if photo_len > 1 else "photo"
+        click.echo(
+            f"Adding {photo_len} {photo_word} to album '{album_query.name}'. Note: Photos may prompt you to confirm this action.",
+            err=True,
+        )
+        try:
+            album_query.add_list(photos)
+        except Exception as e:
+            click.secho(
+                f"Error adding photos to album {add_to_album}: {e}",
+                fg=CLI_COLOR_ERROR,
+                err=True,
+            )
+
+    print_photo_info(photos, cli_json or json_, print_func=click.echo)

osxphotos/cli/repl.py

@@ -0,0 +1,339 @@
+"""repl command for osxphotos CLI"""
+
+import dataclasses
+import os
+import os.path
+import pathlib
+import sys
+import time
+from typing import List
+
+import click
+import photoscript
+from rich import pretty, print
+
+import osxphotos
+from osxphotos._constants import _PHOTOS_4_VERSION
+from osxphotos.photoinfo import PhotoInfo
+from osxphotos.photosdb import PhotosDB
+from osxphotos.pyrepl import embed_repl
+from osxphotos.queryoptions import QueryOptions
+
+from .common import (
+    DB_ARGUMENT,
+    DB_OPTION,
+    DELETED_OPTIONS,
+    QUERY_OPTIONS,
+    get_photos_db,
+    load_uuid_from_file,
+)
+
+
+class IncompatibleQueryOptions(Exception):
+    pass
+
+
+@click.command(name="repl")
+@DB_OPTION
+@click.pass_obj
+@click.pass_context
+@click.option(
+    "--emacs",
+    required=False,
+    is_flag=True,
+    default=False,
+    help="Launch REPL with Emacs keybindings (default is vi bindings)",
+)
+@click.option(
+    "--beta",
+    is_flag=True,
+    default=False,
+    hidden=True,
+    help="Enable beta options.",
+)
+@QUERY_OPTIONS
+@DELETED_OPTIONS
+@click.option("--missing", is_flag=True, help="Search for photos missing from disk.")
+@click.option(
+    "--not-missing",
+    is_flag=True,
+    help="Search for photos present on disk (e.g. not missing).",
+)
+@click.option(
+    "--cloudasset",
+    is_flag=True,
+    help="Search for photos that are part of an iCloud library",
+)
+@click.option(
+    "--not-cloudasset",
+    is_flag=True,
+    help="Search for photos that are not part of an iCloud library",
+)
+@click.option(
+    "--incloud",
+    is_flag=True,
+    help="Search for photos that are in iCloud (have been synched)",
+)
+@click.option(
+    "--not-incloud",
+    is_flag=True,
+    help="Search for photos that are not in iCloud (have not been synched)",
+)
+def repl(ctx, cli_obj, db, emacs, beta, **kwargs):
+    """Run interactive osxphotos REPL shell (useful for debugging, prototyping, and inspecting your Photos library)"""
+    import logging
+
+    from objexplore import explore
+    from photoscript import Album, Photo, PhotosLibrary
+    from rich import inspect as _inspect
+
+    from osxphotos import ExifTool, PhotoInfo, PhotosDB
+    from osxphotos.albuminfo import AlbumInfo
+    from osxphotos.momentinfo import MomentInfo
+    from osxphotos.photoexporter import ExportOptions, ExportResults, PhotoExporter
+    from osxphotos.placeinfo import PlaceInfo
+    from osxphotos.queryoptions import QueryOptions
+    from osxphotos.scoreinfo import ScoreInfo
+    from osxphotos.searchinfo import SearchInfo
+
+    logger = logging.getLogger()
+    logger.disabled = True
+
+    pretty.install()
+    print(f"python version: {sys.version}")
+    print(f"osxphotos version: {osxphotos._version.__version__}")
+    db = db or get_photos_db()
+    photosdb = _load_photos_db(db)
+    # enable beta features if requested
+    if beta:
+        photosdb._beta = beta
+        print("Beta mode enabled")
+    print("Getting photos")
+    tic = time.perf_counter()
+    try:
+        query_options = _query_options_from_kwargs(**kwargs)
+    except IncompatibleQueryOptions:
+        click.echo("Incompatible query options", err=True)
+        click.echo(ctx.obj.group.commands["repl"].get_help(ctx), err=True)
+        sys.exit(1)
+    photos = _query_photos(photosdb, query_options)
+    all_photos = _get_all_photos(photosdb)
+    toc = time.perf_counter()
+    tictoc = toc - tic
+
+    # shortcut for helper functions
+    get_photo = photosdb.get_photo
+    show = _show_photo
+    spotlight = _spotlight_photo
+    get_selected = _get_selected(photosdb)
+    try:
+        selected = get_selected()
+    except Exception:
+        # get_selected sometimes fails
+        selected = []
+
+    def inspect(obj):
+        """inspect object"""
+        return _inspect(obj, methods=True)
+
+    print(f"Found {len(photos)} photos in {tictoc:0.2f} seconds\n")
+    print("The following classes have been imported from osxphotos:")
+    print(
+        "- AlbumInfo, ExifTool, PhotoInfo, PhotoExporter, ExportOptions, ExportResults, PhotosDB, PlaceInfo, QueryOptions, MomentInfo, ScoreInfo, SearchInfo\n"
+    )
+    print("The following variables are defined:")
+    print(f"- photosdb: PhotosDB() instance for {photosdb.library_path}")
+    print(
+        f"- photos: list of PhotoInfo objects for all photos filtered with any query options passed on command line (len={len(photos)})"
+    )
+    print(
+        f"- all_photos: list of PhotoInfo objects for all photos in photosdb, including those in the trash (len={len(all_photos)})"
+    )
+    print(
+        f"- selected: list of PhotoInfo objects for any photos selected in Photos (len={len(selected)})"
+    )
+    print(f"\nThe following functions may be helpful:")
+    print(
+        f"- get_photo(uuid): return a PhotoInfo object for photo with uuid; e.g. get_photo('B13F4485-94E0-41CD-AF71-913095D62E31')"
+    )
+    print(
+        f"- get_selected(); return list of PhotoInfo objects for photos selected in Photos"
+    )
+    print(
+        f"- show(photo): open a photo object in the default viewer; e.g. show(selected[0])"
+    )
+    print(
+        f"- show(path): open a file at path in the default viewer; e.g. show('/path/to/photo.jpg')"
+    )
+    print(f"- spotlight(photo): open a photo and spotlight it in Photos")
+    # print(
+    #     f"- help(object): print help text including list of methods for object; for example, help(PhotosDB)"
+    # )
+    print(
+        f"- inspect(object): print information about an object; e.g. inspect(PhotoInfo)"
+    )
+    print(
+        f"- explore(object): interactively explore an object with objexplore; e.g. explore(PhotoInfo)"
+    )
+    print(f"- q, quit, quit(), exit, exit(): exit this interactive shell\n")
+
+    embed_repl(
+        globals=globals(),
+        locals=locals(),
+        history_filename=str(pathlib.Path.home() / ".osxphotos_repl_history"),
+        quit_words=["q", "quit", "exit"],
+        vi_mode=not emacs,
+    )
+
+
+def _show_photo(photo: PhotoInfo):
+    """open image with default image viewer
+
+    Note: This is for debugging only -- it will actually open any filetype which could
+    be very, very bad.
+
+    Args:
+        photo: PhotoInfo object or a path to a photo on disk
+    """
+    photopath = photo.path if isinstance(photo, osxphotos.PhotoInfo) else photo
+
+    if not os.path.isfile(photopath):
+        return f"'{photopath}' does not appear to be a valid photo path"
+
+    os.system(f"open '{photopath}'")
+
+
+def _load_photos_db(dbpath):
+    print("Loading database")
+    tic = time.perf_counter()
+    photosdb = osxphotos.PhotosDB(dbfile=dbpath, verbose=print)
+    toc = time.perf_counter()
+    tictoc = toc - tic
+    print(f"Done: took {tictoc:0.2f} seconds")
+    return photosdb
+
+
+def _get_all_photos(photosdb):
+    """get list of all photos in photosdb"""
+    photos = photosdb.photos(images=True, movies=True)
+    photos.extend(photosdb.photos(images=True, movies=True, intrash=True))
+    return photos
+
+
+def _get_selected(photosdb):
+    """get list of PhotoInfo objects for photos selected in Photos"""
+
+    def get_selected():
+        selected = photoscript.PhotosLibrary().selection
+        if not selected:
+            return []
+        return photosdb.photos(uuid=[p.uuid for p in selected])
+
+    return get_selected
+
+
+def _spotlight_photo(photo: PhotoInfo):
+    photo_ = photoscript.Photo(photo.uuid)
+    photo_.spotlight()
+
+
+def _query_options_from_kwargs(**kwargs) -> QueryOptions:
+    """Validate query options and create a QueryOptions instance"""
+    # sanity check input args
+    nonexclusive = [
+        "keyword",
+        "person",
+        "album",
+        "folder",
+        "name",
+        "uuid",
+        "uuid_from_file",
+        "edited",
+        "external_edit",
+        "uti",
+        "has_raw",
+        "from_date",
+        "to_date",
+        "from_time",
+        "to_time",
+        "label",
+        "is_reference",
+        "query_eval",
+        "query_function",
+        "min_size",
+        "max_size",
+        "regex",
+        "selected",
+        "exif",
+        "duplicate",
+    ]
+    exclusive = [
+        ("favorite", "not_favorite"),
+        ("hidden", "not_hidden"),
+        ("missing", "not_missing"),
+        ("only_photos", "only_movies"),
+        ("burst", "not_burst"),
+        ("live", "not_live"),
+        ("cloudasset", "not_cloudasset"),
+        ("incloud", "not_incloud"),
+        ("portrait", "not_portrait"),
+        ("screenshot", "not_screenshot"),
+        ("slow_mo", "not_slow_mo"),
+        ("time_lapse", "not_time_lapse"),
+        ("hdr", "not_hdr"),
+        ("selfie", "not_selfie"),
+        ("panorama", "not_panorama"),
+        ("deleted", "deleted_only"),
+        ("shared", "not_shared"),
+        ("has_comment", "no_comment"),
+        ("has_likes", "no_likes"),
+        ("in_album", "not_in_album"),
+        ("location", "no_location"),
+    ]
+    # print help if no non-exclusive term or a double exclusive term is given
+    # TODO: add option to validate requiring at least one query arg
+    if any(all([kwargs[b], kwargs[n]]) for b, n in exclusive) or any(
+        [
+            all([any(kwargs["title"]), kwargs["no_title"]]),
+            all([any(kwargs["description"]), kwargs["no_description"]]),
+            all([any(kwargs["place"]), kwargs["no_place"]]),
+        ]
+    ):
+        raise IncompatibleQueryOptions
+
+    # actually have something to query
+    include_photos = True
+    include_movies = True  # default searches for everything
+    if kwargs["only_movies"]:
+        include_photos = False
+    if kwargs["only_photos"]:
+        include_movies = False
+
+    # load UUIDs if necessary and append to any uuids passed with --uuid
+    uuid = None
+    if kwargs["uuid_from_file"]:
+        uuid_list = list(kwargs["uuid"])  # Click option is a tuple
+        uuid_list.extend(load_uuid_from_file(kwargs["uuid_from_file"]))
+        uuid = tuple(uuid_list)
+
+    query_fields = [field.name for field in dataclasses.fields(QueryOptions)]
+    query_dict = {field: kwargs.get(field) for field in query_fields}
+    query_dict["photos"] = include_photos
+    query_dict["movies"] = include_movies
+    query_dict["uuid"] = uuid
+    return QueryOptions(**query_dict)
+
+
+def _query_photos(photosdb: PhotosDB, query_options: QueryOptions) -> List:
+    """Query photos given a QueryOptions instance"""
+    try:
+        photos = photosdb.query(query_options)
+    except ValueError as e:
+        if "Invalid query_eval CRITERIA:" not in str(e):
+            raise ValueError(e) from e
+        msg = str(e).split(":")[1]
+        raise click.BadOptionUsage(
+            "query_eval", f"Invalid query-eval CRITERIA: {msg}"
+        ) from e
+
+    return photos

osxphotos/cli/snap_diff.py

@@ -0,0 +1,157 @@
+"""snap/diff commands for osxphotos CLI"""
+
+import datetime
+import os
+import pathlib
+import shutil
+import subprocess
+
+import click
+from rich.console import Console
+from rich.syntax import Syntax
+
+import osxphotos
+
+from .common import DB_OPTION, OSXPHOTOS_SNAPSHOT_DIR, get_photos_db, verbose_print
+
+
+@click.command(name="snap")
+@click.pass_obj
+@click.pass_context
+@DB_OPTION
+def snap(ctx, cli_obj, db):
+    """Create snapshot of Photos database to use with diff command
+
+    Snapshots only the database files, not the entire library. If OSXPHOTOS_SNAPSHOT
+    environment variable is defined, will use that as snapshot directory, otherwise
+    uses '/private/tmp/osxphotos_snapshots'
+
+    Works only on Photos library versions since Catalina (10.15) or newer.
+    """
+
+    db = get_photos_db(db, cli_obj.db)
+    db_path = pathlib.Path(db)
+    if db_path.is_file():
+        # assume it's the sqlite file
+        db_path = db_path.parent.parent
+    db_path = db_path / "database"
+
+    db_folder = os.environ.get("OSXPHOTOS_SNAPSHOT", OSXPHOTOS_SNAPSHOT_DIR)
+    if not os.path.isdir(db_folder):
+        click.echo(f"Creating snapshot folder: '{db_folder}'")
+        os.mkdir(db_folder)
+
+    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+    destination_path = pathlib.Path(db_folder) / timestamp
+
+    # get all the sqlite files including the write ahead log if any
+    files = db_path.glob("*.sqlite*")
+    os.makedirs(destination_path)
+    fu = osxphotos.fileutil.FileUtil()
+    count = 0
+    for file in files:
+        if file.is_file():
+            fu.copy(file, destination_path)
+            count += 1
+
+    print(f"Copied {count} files from {db_path} to {destination_path}")
+
+
+@click.command(name="diff")
+@click.pass_obj
+@click.pass_context
+@DB_OPTION
+@click.option(
+    "--raw-output",
+    "-r",
+    is_flag=True,
+    default=False,
+    help="Print raw output (don't use syntax highlighting).",
+)
+@click.option(
+    "--style",
+    "-s",
+    metavar="STYLE",
+    nargs=1,
+    default="monokai",
+    help="Specify style/theme for syntax highlighting. "
+    "Theme may be any valid pygments style (https://pygments.org/styles/). "
+    "Default is 'monokai'.",
+)
+@click.argument("db2", nargs=-1, type=click.Path(exists=True))
+@click.option("--verbose", "-V", "verbose", is_flag=True, help="Print verbose output.")
+def diff(ctx, cli_obj, db, raw_output, style, db2, verbose):
+    """Compare two Photos databases and print out differences
+
+    To use the diff command, you'll need to install sqldiff via homebrew:
+
+     - Install homebrew (https://brew.sh/) if not already installed
+
+     - Install sqldiff: `brew install sqldiff`
+
+    When run with no arguments, compares the current Photos library to the
+    most recent snapshot in the the OSXPHOTOS_SNAPSHOT directory.
+
+    If run with the --db option, compares the library specified by --db to the
+    most recent snapshot in the the OSXPHOTOS_SNAPSHOT directory.
+
+    If run with just the DB2 argument, compares the current Photos library to
+    the database specified by the DB2 argument.
+
+    If run with both the --db option and the DB2 argument, compares the
+    library specified by --db to the database specified by DB2
+
+    See also `osxphotos snap`
+
+    If the OSXPHOTOS_SNAPSHOT environment variable is not set, will use
+    '/private/tmp/osxphotos_snapshots'
+
+    Works only on Photos library versions since Catalina (10.15) or newer.
+    """
+
+    verbose_ = verbose_print(verbose, rich=True)
+
+    sqldiff = shutil.which("sqldiff")
+    if not sqldiff:
+        click.echo(
+            "sqldiff not found; install via homebrew (https://brew.sh/): `brew install sqldiff`"
+        )
+        ctx.exit(2)
+    verbose_(f"sqldiff found at '{sqldiff}'")
+
+    db = get_photos_db(db, cli_obj.db)
+    db_path = pathlib.Path(db)
+    if db_path.is_file():
+        # assume it's the sqlite file
+        db_path = db_path.parent.parent
+    db_path = db_path / "database"
+    db_1 = db_path / "photos.sqlite"
+
+    if db2:
+        db_2 = pathlib.Path(db2[0])
+    else:
+        # get most recent snapshot
+        db_folder = os.environ.get("OSXPHOTOS_SNAPSHOT", OSXPHOTOS_SNAPSHOT_DIR)
+        verbose_(f"Using snapshot folder: '{db_folder}'")
+        folders = sorted([f for f in pathlib.Path(db_folder).glob("*") if f.is_dir()])
+        folder_2 = folders[-1]
+        db_2 = folder_2 / "Photos.sqlite"
+
+    if not db_1.exists():
+        print(f"database file {db_1} missing")
+    if not db_2.exists():
+        print(f"database file {db_2} missing")
+
+    verbose_(f"Comparing databases {db_1} and {db_2}")
+
+    diff_proc = subprocess.Popen([sqldiff, db_2, db_1], stdout=subprocess.PIPE)
+    console = Console()
+    for line in iter(diff_proc.stdout.readline, b""):
+        line = line.decode("UTF-8").rstrip()
+        if raw_output:
+            print(line)
+        else:
+            syntax = Syntax(
+                line, "sql", theme=style, line_numbers=False, code_width=1000
+            )
+            console.print(syntax)

osxphotos/cli/tutorial.py

@@ -0,0 +1,46 @@
+"""tutorial command for osxphotos CLI"""
+
+import io
+import pathlib
+
+import click
+from rich.console import Console
+from rich.markdown import Markdown
+
+from .help import strip_html_comments, strip_md_links
+
+
+@click.command(name="tutorial")
+@click.argument(
+    "WIDTH",
+    nargs=-1,
+    type=click.INT,
+)
+@click.pass_obj
+@click.pass_context
+def tutorial(ctx, cli_obj, width):
+    """Display osxphotos tutorial."""
+    width = width[0] if width else 100
+    click.echo_via_pager(tutorial_help(width=width))
+
+
+def tutorial_help(width=78):
+    """Return formatted string for tutorial"""
+    sio = io.StringIO()
+    console = Console(file=sio, force_terminal=True, width=width)
+    help_md = get_tutorial_text()
+    help_md = strip_html_comments(help_md)
+    help_md = strip_md_links(help_md)
+    console.print(Markdown(help_md))
+    help_str = sio.getvalue()
+    sio.close()
+    return help_str
+
+
+def get_tutorial_text():
+    """Load tutorial text from file"""
+    # TODO: would be better to use importlib.abc.ResourceReader but I can't find a single example of how to do this
+    help_file = pathlib.Path(__file__).parent / "../tutorial.md"
+    with open(help_file, "r") as fd:
+        md = fd.read()
+    return md

osxphotos/cli/uuid.py

@@ -0,0 +1,26 @@
+"""uuid command for osxphotos CLI"""
+
+import click
+import photoscript
+
+
+@click.command(name="uuid")
+@click.pass_obj
+@click.pass_context
+@click.option(
+    "--filename",
+    "-f",
+    required=False,
+    is_flag=True,
+    default=False,
+    help="Include filename of selected photos in output",
+)
+def uuid(ctx, cli_obj, filename):
+    """Print out unique IDs (UUID) of photos selected in Photos
+
+    Prints outs UUIDs in form suitable for --uuid-from-file and --skip-uuid-from-file
+    """
+    for photo in photoscript.PhotosLibrary().selection:
+        if filename:
+            print(f"# {photo.filename}")
+        print(photo.uuid)

osxphotos/configoptions.py

@@ -1,4 +1,5 @@
 """ ConfigOptions class to load/save config settings for osxphotos CLI """
+import bitmath
 import toml
 
 __all__ = [
@@ -178,6 +179,10 @@ class ConfigOptions:
         data = {}
         for attr in sorted(self._attrs.keys()):
             val = getattr(self, attr)
+
+            if isinstance(val, bitmath.Bitmath):
+                val = int(val.to_Byte())
+
             if val in [False, ()]:
                 val = None
             else:

osxphotos/exiftool.py

@@ -69,6 +69,8 @@ def unescape_str(s):
     """unescape an HTML string returned by exiftool -E"""
     if type(s) != str:
         return s
+    # avoid " in values which result in json.loads() throwing an exception, #636
+    s = s.replace(""", '\\"')
     return html.unescape(s)
 
 
@@ -105,7 +107,8 @@ class _ExifToolProc:
 
     def __init__(self, exiftool=None):
         """construct _ExifToolProc singleton object or return instance of already created object
-        exiftool: optional path to exiftool binary (if not provided, will search path to find it)"""
+        exiftool: optional path to exiftool binary (if not provided, will search path to find it)
+        """
 
         if hasattr(self, "_process_running") and self._process_running:
             # already running
@@ -115,7 +118,6 @@ class _ExifToolProc:
                     f"ignoring exiftool={exiftool}"
                 )
             return
-
         self._process_running = False
         self._exiftool = exiftool or get_exiftool_path()
         self._start_proc()
@@ -147,6 +149,9 @@ class _ExifToolProc:
             return
 
         # open exiftool process
+        # make sure /usr/bin at start of path so exiftool can find xattr (see #636)
+        env = os.environ.copy()
+        env["PATH"] = f'/usr/bin/:{env["PATH"]}'
         self._process = subprocess.Popen(
             [
                 self._exiftool,
@@ -163,6 +168,7 @@ class _ExifToolProc:
             stdin=subprocess.PIPE,
             stdout=subprocess.PIPE,
             stderr=subprocess.STDOUT,
+            env=env,
         )
         self._process_running = True
 
@@ -362,6 +368,7 @@ class ExifTool:
         error = "" if error == b"" else error.decode("utf-8")
         self.warning = warning
         self.error = error
+
         return output[:-EXIFTOOL_STAYOPEN_EOF_LEN], warning, error
 
     @property
@@ -393,6 +400,7 @@ class ExifTool:
         except Exception as e:
             # will fail with some commands, e.g --ext AVI which produces
             # 'No file with specified extension' instead of json
+            logging.warning(f"error loading json returned by exiftool: {e} {json_str}")
             return dict()
         exifdict = exifdict[0]
         if not tag_groups:

osxphotos/export_db.py

@@ -50,6 +50,16 @@ class ExportDB:
         self._perform_db_maintenace(self._conn)
         self._insert_run_info()
 
+    @property
+    def path(self):
+        """returns path to export database"""
+        return self._dbfile
+
+    @property
+    def export_dir(self):
+        """returns path to export directory"""
+        return self._path
+
     def get_file_record(self, filename: Union[pathlib.Path, str]) -> "ExportRecord":
         """get info for filename and uuid
 
@@ -566,7 +576,14 @@ class ExportDBInMemory(ExportDB):
     modifying the on-disk version
     """
 
-    def __init__(self, dbfile, export_dir):
+    def __init__(self, dbfile: str, export_dir: str):
+        """ "Initialize ExportDBInMemory
+
+        Args:
+            dbfile (str): path to database file
+            export_dir (str): path to export directory
+            write_back (bool): whether to write changes back to disk when closing; if False (default), changes are not written to disk
+        """
         self._dbfile = dbfile or f"./{OSXPHOTOS_EXPORT_DB}"
         # export_dir is required as all files referenced by get_/set_uuid_for_file will be converted to
         # relative paths to this path
@@ -576,6 +593,39 @@ class ExportDBInMemory(ExportDB):
         self._conn = self._open_export_db(self._dbfile)
         self._insert_run_info()
 
+    def write_to_disk(self):
+        """Write changes from in-memory database back to disk"""
+
+        # dump the database
+        conn = self._conn
+        conn.commit()
+        dbdump = self._dump_db(conn)
+
+        # cleanup the old on-disk database
+        # also unlink the wal and shm files if needed
+        dbfile = pathlib.Path(self._dbfile)
+        if dbfile.exists():
+            dbfile.unlink()
+        wal = dbfile.with_suffix(".db-wal")
+        if wal.exists():
+            wal.unlink()
+        shm = dbfile.with_suffix(".db-shm")
+        if shm.exists():
+            shm.unlink()
+
+        conn_on_disk = sqlite3.connect(str(dbfile))
+        conn_on_disk.cursor().executescript(dbdump.read())
+        conn_on_disk.commit()
+        conn_on_disk.close()
+
+    def close(self):
+        """close the database connection"""
+        try:
+            if self._conn:
+                self._conn.close()
+        except Error as e:
+            logging.warning(e)
+
     def _open_export_db(self, dbfile):
         """open export database and return a db connection
         returns: connection to the database
@@ -588,21 +638,13 @@ class ExportDBInMemory(ExportDB):
             self.was_created = True
             self.was_upgraded = ()
         else:
-            try:
-                conn = sqlite3.connect(dbfile)
-            except Error as e:
-                logging.warning(e)
-                raise e from e
-
-            tempfile = StringIO()
-            for line in conn.iterdump():
-                tempfile.write("%s\n" % line)
+            conn = sqlite3.connect(dbfile)
+            dbdump = self._dump_db(conn)
             conn.close()
-            tempfile.seek(0)
 
-            # Create a database in memory and import from tempfile
+            # Create a database in memory and import from the dump
             conn = sqlite3.connect(":memory:")
-            conn.cursor().executescript(tempfile.read())
+            conn.cursor().executescript(dbdump.read())
             conn.commit()
             self.was_created = False
             version_info = self._get_database_version(conn)
@@ -625,6 +667,21 @@ class ExportDBInMemory(ExportDB):
 
         return conn
 
+    def _dump_db(self, conn: sqlite3.Connection) -> StringIO:
+        """dump sqlite db to a string buffer"""
+        dbdump = StringIO()
+        for line in conn.iterdump():
+            dbdump.write("%s\n" % line)
+        dbdump.seek(0)
+        return dbdump
+
+    def __del__(self):
+        """close the database connection"""
+        try:
+            self.close()
+        except Error as e:
+            pass
+
 
 class ExportDBTemp(ExportDBInMemory):
     """Temporary in-memory version of ExportDB"""

osxphotos/export_db_utils.py

@@ -1,17 +1,18 @@
 """ Utility functions for working with export_db """
 
 
-import pathlib
-import sqlite3
-from typing import Optional, Tuple, Union
 import datetime
 import os
+import pathlib
+import sqlite3
+from typing import Callable, Optional, Tuple, Union
 
 import toml
 from rich import print
 
 from ._constants import OSXPHOTOS_EXPORT_DB
 from ._version import __version__
+from .utils import noop
 from .export_db import OSXPHOTOS_EXPORTDB_VERSION, ExportDB
 from .fileutil import FileUtil
 from .photosdb import PhotosDB
@@ -57,7 +58,7 @@ def export_db_vacuum(dbfile: Union[str, pathlib.Path]) -> None:
 def export_db_update_signatures(
     dbfile: Union[str, pathlib.Path],
     export_dir: Union[str, pathlib.Path],
-    verbose: bool = False,
+    verbose_: Callable = noop,
     dry_run: bool = False,
 ) -> Tuple[int, int]:
     """Update signatures for all files found in the export database to match what's on disk
@@ -78,13 +79,11 @@ def export_db_update_signatures(
         filepath = export_dir / filepath
         if not os.path.exists(filepath):
             skipped += 1
-            if verbose:
-                print(f"[dark_orange]Skipping missing file[/dark_orange]: '{filepath}'")
+            verbose_(f"[dark_orange]Skipping missing file[/dark_orange]: '{filepath}'")
             continue
         updated += 1
         file_sig = fileutil.file_sig(filepath)
-        if verbose:
-            print(f"[green]Updating signature for[/green]: '{filepath}'")
+        verbose_(f"[green]Updating signature for[/green]: '{filepath}'")
         if not dry_run:
             c.execute(
                 "UPDATE export_data SET dest_mode = ?, dest_size = ?, dest_mtime = ? WHERE filepath_normalized = ?;",
@@ -129,7 +128,7 @@ def export_db_save_config_to_file(
 def export_db_check_signatures(
     dbfile: Union[str, pathlib.Path],
     export_dir: Union[str, pathlib.Path],
-    verbose: bool = False,
+    verbose_: Callable = noop,
 ) -> Tuple[int, int, int]:
     """Check signatures for all files found in the export database to verify what matches the on disk files
 
@@ -151,19 +150,16 @@ def export_db_check_signatures(
         filepath = export_dir / filepath
         if not filepath.exists():
             skipped += 1
-            if verbose:
-                print(f"[dark_orange]Skipping missing file[/dark_orange]: '{filepath}'")
+            verbose_(f"[dark_orange]Skipping missing file[/dark_orange]: '{filepath}'")
             continue
         file_sig = fileutil.file_sig(filepath)
         file_rec = exportdb.get_file_record(filepath)
         if file_rec.dest_sig == file_sig:
             matched += 1
-            if verbose:
-                print(f"[green]Signatures matched[/green]: '{filepath}'")
+            verbose_(f"[green]Signatures matched[/green]: '{filepath}'")
         else:
             notmatched += 1
-            if verbose:
-                print(f"[deep_pink3]Signatures do not match[/deep_pink3]: '{filepath}'")
+            verbose_(f"[deep_pink3]Signatures do not match[/deep_pink3]: '{filepath}'")
 
     return (matched, notmatched, skipped)
 
@@ -171,7 +167,7 @@ def export_db_check_signatures(
 def export_db_touch_files(
     dbfile: Union[str, pathlib.Path],
     export_dir: Union[str, pathlib.Path],
-    verbose: bool = False,
+    verbose_: Callable = noop,
     dry_run: bool = False,
 ) -> Tuple[int, int, int]:
     """Touch files on disk to match the Photos library created date
@@ -183,8 +179,8 @@ def export_db_touch_files(
     # open and close exportdb to ensure it gets migrated
     exportdb = ExportDB(dbfile, export_dir)
     upgraded = exportdb.was_upgraded
-    if upgraded and verbose:
-        print(
+    if upgraded:
+        verbose_(
             f"Upgraded export database {dbfile} from version {upgraded[0]} to {upgraded[1]}"
         )
     exportdb.close()
@@ -204,7 +200,6 @@ def export_db_touch_files(
         # in the mean time, photos_db_path = None will use the default library
         photos_db_path = None
 
-    verbose_ = print if verbose else lambda *args, **kwargs: None
     photosdb = PhotosDB(dbfile=photos_db_path, verbose=verbose_)
     exportdb = ExportDB(dbfile, export_dir)
     c.execute(
@@ -223,19 +218,17 @@ def export_db_touch_files(
         dest_size = row[4]
         if not filepath.exists():
             skipped += 1
-            if verbose:
-                print(
-                    f"[dark_orange]Skipping missing file (not in export directory)[/dark_orange]: '{filepath}'"
-                )
+            verbose_(
+                f"[dark_orange]Skipping missing file (not in export directory)[/dark_orange]: '{filepath}'"
+            )
             continue
 
         photo = photosdb.get_photo(uuid)
         if not photo:
             skipped += 1
-            if verbose:
-                print(
-                    f"[dark_orange]Skipping missing photo (did not find in Photos Library)[/dark_orange]: '{filepath}' ({uuid})"
-                )
+            verbose_(
+                f"[dark_orange]Skipping missing photo (did not find in Photos Library)[/dark_orange]: '{filepath}' ({uuid})"
+            )
             continue
 
         ts = int(photo.date.timestamp())
@@ -243,18 +236,16 @@ def export_db_touch_files(
         mtime = stat.st_mtime
         if mtime == ts:
             not_touched += 1
-            if verbose:
-                print(
-                    f"[green]Skipping file (timestamp matches)[/green]: '{filepath}' [dodger_blue1]{isotime_from_ts(ts)} ({ts})[/dodger_blue1]"
-                )
+            verbose_(
+                f"[green]Skipping file (timestamp matches)[/green]: '{filepath}' [dodger_blue1]{isotime_from_ts(ts)} ({ts})[/dodger_blue1]"
+            )
             continue
 
         touched += 1
-        if verbose:
-            print(
-                f"[deep_pink3]Touching file[/deep_pink3]: '{filepath}' "
-                f"[dodger_blue1]{isotime_from_ts(mtime)} ({mtime}) -> {isotime_from_ts(ts)} ({ts})[/dodger_blue1]"
-            )
+        verbose_(
+            f"[deep_pink3]Touching file[/deep_pink3]: '{filepath}' "
+            f"[dodger_blue1]{isotime_from_ts(mtime)} ({mtime}) -> {isotime_from_ts(ts)} ({ts})[/dodger_blue1]"
+        )
 
         if not dry_run:
             os.utime(str(filepath), (ts, ts))

osxphotos/photoexporter.py

@@ -560,11 +560,15 @@ class PhotoExporter:
         touch_results = []
         for touch_file in set(touch_files):
             ts = int(self.photo.date.timestamp())
-            stat = os.stat(touch_file)
-            if stat.st_mtime != ts:
-                if not options.dry_run:
+            try:
+                stat = os.stat(touch_file)
+                if stat.st_mtime != ts:
                     fileutil.utime(touch_file, (ts, ts))
-                touch_results.append(touch_file)
+                    touch_results.append(touch_file)
+            except FileNotFoundError as e:
+                # ignore errors if in dry_run as file may not be present
+                if not options.dry_run:
+                    raise e from e
         return ExportResults(touched=touch_results)
 
     def _get_edited_filename(self, original_filename):
@@ -669,8 +673,8 @@ class PhotoExporter:
 
         if file_record.export_options != options.bit_flags:
             # exporting with different set of options (e.g. exiftool), should update
-            # need to check this before exiftool in case exiftool options are different 
-            # and export database is missing; this will always be True if database is missing 
+            # need to check this before exiftool in case exiftool options are different
+            # and export database is missing; this will always be True if database is missing
             # as it'll be None and bit_flags will be an int
             return True
 

osxphotos/photoinfo.py

@@ -54,7 +54,7 @@ from .scoreinfo import ScoreInfo
 from .searchinfo import SearchInfo
 from .text_detection import detect_text
 from .uti import get_preferred_uti_extension, get_uti_for_extension
-from .utils import _debug, _get_resource_loc, list_directory
+from .utils import _debug, _get_resource_loc, list_directory, _debug
 
 __all__ = ["PhotoInfo", "PhotoInfoNone"]
 
@@ -588,7 +588,7 @@ class PhotoInfo:
     @property
     def ismissing(self):
         """returns true if photo is missing from disk (which means it's not been downloaded from iCloud)
-        
+
         NOTE:   the photos.db database uses an asynchrounous write-ahead log so changes in Photos
                 do not immediately get written to disk. In particular, I've noticed that downloading
                 an image from the cloud does not force the database to be updated until something else

osxphotos/photosdb/photosdb.py

@@ -3279,27 +3279,6 @@ class PhotosDB:
         if options.to_time:
             photos = [p for p in photos if p.date.time() <= options.to_time]
 
-        if options.burst_photos:
-            # add the burst_photos to the export set
-            photos_burst = [p for p in photos if p.burst]
-            for burst in photos_burst:
-                if options.missing_bursts:
-                    # include burst photos that are missing
-                    photos.extend(burst.burst_photos)
-                else:
-                    # don't include missing burst images (these can't be downloaded with AppleScript)
-                    photos.extend([p for p in burst.burst_photos if not p.ismissing])
-
-            # remove duplicates as each burst photo in the set that's selected would
-            # result in the entire set being added above
-            # can't use set() because PhotoInfo not hashable
-            seen_uuids = {}
-            for p in photos:
-                if p.uuid in seen_uuids:
-                    continue
-                seen_uuids[p.uuid] = p
-            photos = list(seen_uuids.values())
-
         if name:
             # search filename fields for text
             # if more than one, find photos with all title values in filename
@@ -3450,6 +3429,28 @@ class PhotosDB:
             for function in options.function:
                 photos = function[0](photos)
 
+        # burst should be checked last, ref #640
+        if options.burst_photos:
+            # add the burst_photos to the export set
+            photos_burst = [p for p in photos if p.burst]
+            for burst in photos_burst:
+                if options.missing_bursts:
+                    # include burst photos that are missing
+                    photos.extend(burst.burst_photos)
+                else:
+                    # don't include missing burst images (these can't be downloaded with AppleScript)
+                    photos.extend([p for p in burst.burst_photos if not p.ismissing])
+
+            # remove duplicates as each burst photo in the set that's selected would
+            # result in the entire set being added above
+            # can't use set() because PhotoInfo not hashable
+            seen_uuids = {}
+            for p in photos:
+                if p.uuid in seen_uuids:
+                    continue
+                seen_uuids[p.uuid] = p
+            photos = list(seen_uuids.values())
+
         return photos
 
     def execute(self, sql):

osxphotos/sqlgrep.py

@@ -29,7 +29,7 @@ def sqlgrep(
     flags = re.IGNORECASE if ignore_case else 0
     try:
         with sqlite3.connect(f"file:{filename}?mode=ro", uri=True) as conn:
-            regex = re.compile(r"(" + pattern + r")", flags=flags)
+            regex = re.compile(f'({pattern})', flags=flags)
             filename_header = f"{filename}: " if print_filename else ""
             conn.row_factory = sqlite3.Row
             cursor = conn.cursor()
@@ -54,4 +54,4 @@ def sqlgrep(
                                 field_value,
                             ]
     except sqlite3.DatabaseError as e:
-        raise sqlite3.DatabaseError(f"{filename}: {e}")
+        raise sqlite3.DatabaseError(f"{filename}: {e}") from e

requirements.txt

@@ -1,26 +1,25 @@
-Click>=8.0.1,<9.0
-Mako>=1.1.4,<1.2.0
-PyYAML>=5.4.1,<6.0.0
 bitmath>=1.3.3.1,<1.4.0.0
 bpylist2==3.0.2
-dataclasses==0.7;python_version<'3.7'
+Click>=8.0.4,<9.0
+Mako>=1.1.4,<1.2.0
 more-itertools>=8.8.0,<9.0.0
-objexplore>=1.5.5,<1.6.0
+objexplore>=1.6.3,<2.0.0
 osxmetadata>=0.99.34,<1.0.0
 pathvalidate>=2.4.1,<2.5.0
 photoscript>=0.1.4,<0.2.0
 ptpython>=3.0.20,<3.1.0
 pyobjc-core>=7.3,<9.0
-pyobjc-framework-AVFoundation>=7.3,<9.0
 pyobjc-framework-AppleScriptKit>=7.3,<9.0
 pyobjc-framework-AppleScriptObjC>=7.3,<9.0
+pyobjc-framework-AVFoundation>=7.3,<9.0
 pyobjc-framework-Cocoa>=7.3,<9.0
 pyobjc-framework-CoreServices>=7.2,<9.0
 pyobjc-framework-Metal>=7.3,<9.0
 pyobjc-framework-Photos>=7.3,<9.0
 pyobjc-framework-Quartz>=7.3,<9.0
 pyobjc-framework-Vision>=7.3,<9.0
-rich>=10.6.0,<=11.0.0
+PyYAML>=5.4.1,<6.0.0
+rich>=11.2.0,<12.0.0
 textx>=2.3.0,<2.4.0
 toml>=0.10.2,<0.11.0
 wurlitzer>=2.1.0,<2.2.0

setup.py

@@ -67,21 +67,19 @@ setup(
         "Intended Audience :: Developers",
         "License :: OSI Approved :: MIT License",
         "Operating System :: MacOS :: MacOS X",
-        "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
         "Programming Language :: Python :: 3.10",
         "Topic :: Software Development :: Libraries :: Python Modules",
     ],
     install_requires=[
-        "Click>=8.0.1,<9.0",
+        "Click>=8.0.4,<9.0",
         "Mako>=1.1.4,<1.2.0",
         "PyYAML>=5.4.1,<5.5.0",
         "bitmath>=1.3.3.1,<1.4.0.0",
         "bpylist2==3.0.2",
-        "dataclasses==0.7;python_version<'3.7'",
         "more-itertools>=8.8.0,<9.0.0",
-        "objexplore>=1.5.5,<1.6.0",
+        "objexplore>=1.6.3,<2.0.0",
         "osxmetadata>=0.99.34,<1.0.0",
         "pathvalidate>=2.4.1,<3.0.0",
         "photoscript>=0.1.4,<0.2.0",
@@ -96,11 +94,11 @@ setup(
         "pyobjc-framework-Photos>=7.3,<9.0",
         "pyobjc-framework-Quartz>=7.3,<9.0",
         "pyobjc-framework-Vision>=7.3,<9.0",
-        "rich>=10.6.0,<=11.0.0",
+        "rich>=11.2.0,<12.0.0",
         "textx>=2.3.0,<3.0.0",
         "toml>=0.10.2,<0.11.0",
         "wurlitzer>=2.1.0,<3.0.0",
     ],
-    entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli"]},
+    entry_points={"console_scripts": ["osxphotos=osxphotos.__main__:cli_main"]},
     include_package_data=True,
 )

tests/README.md

@@ -1,14 +1,18 @@
 # Tests for osxphotos #
 
 ## Running Tests ##
-Tests require pytest and pytest-mock:
-`pip install pytest`
-`pip install pytest-mock`
+To set up a dev environment to work on osxphotos code or run tests follow these steps.  This assumes you have python 3.7 or later installed.  If you need to install python, you can do so with the XCode command lines tools (`xcode-select --install`) or from [python.org](https://www.python.org/downloads/macos/).
+
+- `git clone git@github.com:RhetTbull/osxphotos.git`
+- `cd osxphotos`
+- `python3 -m venv venv`
+- `source venv/bin/activate`
+- `python3 -m pip install -r dev_requirements.txt`
+- `python3 -m pip install -e .`
 
 To run the tests, do the following from the main source folder:
-`python -m pytest tests/`
+`python3 -m pytest tests/`
 
-Running the tests this way allows the library to be tested without installing it.
 
 ## Skipped Tests ##
 A few tests will look for certain environment variables to determine if they should run.

tests/conftest.py

@@ -3,10 +3,9 @@ import os
 import pathlib
 
 import pytest
+from applescript import AppleScript
 from photoscript.utils import ditto
 
-import osxphotos
-from applescript import AppleScript
 from osxphotos.exiftool import _ExifToolProc
 
 
@@ -124,3 +123,12 @@ def copy_photos_library_to_path(photos_library_path: str, dest_path: str) -> str
     """Copy a photos library to a folder"""
     ditto(photos_library_path, dest_path)
     return dest_path
+
+
+@pytest.fixture(scope="session", autouse=True)
+def delete_crash_logs():
+    """Delete left over crash logs from tests that were supposed to crash"""
+    yield
+    path = pathlib.Path(os.getcwd()) / "osxphotos_crash.log"
+    if path.is_file():
+        path.unlink()

tests/test___all__.py

@@ -1,25 +0,0 @@
-import re
-import sys
-
-from os import walk
-from collections import Counter
-
-
-FILE_PATTERN = "^(?!_).*\.py$"
-SOUCE_CODE_ROOT = "osxphotos"
-
-def create_module_name(dirpath: str, filename: str) -> str:
-    prefix = dirpath[dirpath.rfind(SOUCE_CODE_ROOT):].replace('/', '.')
-    return f"{prefix}.{filename}".replace(".py", "")
-
-
-def test_check_duplicate():
-    for dirpath, dirnames, filenames in walk(SOUCE_CODE_ROOT):
-        print("\n", sys.modules)
-        for filename in filenames:
-            if re.search(FILE_PATTERN, filename):
-                module = create_module_name(dirpath, filename)
-                if module in sys.modules:
-                    all_list = sys.modules[module].__all__
-                    all_set = set(all_list)
-                    assert Counter(all_list) == Counter(all_set)

tests/test_catalina_10_15_7.py

@@ -1405,10 +1405,9 @@ def test_exiftool_newlines_in_description(photosdb):
     assert exif["EXIF:ImageDescription"].find("\n") > 0
 
 
-@pytest.mark.skip(SKIP_TEST, reason="Not yet implemented")
+@pytest.mark.skip(reason="Test not yet implemented")
 def test_duplicates_1(photosdb):
     # test photo has duplicates
-
     photo = photosdb.get_photo(uuid=UUID_DICT["duplicates"])
     assert len(photo.duplicates) == 1
     assert photo.duplicates[0].uuid == UUID_DUPLICATE

tests/test_exiftool.py

@@ -1,5 +1,8 @@
+import json
+
 import pytest
-from osxphotos.exiftool import get_exiftool_path
+
+from osxphotos.exiftool import get_exiftool_path, unescape_str
 
 TEST_FILE_ONE_KEYWORD = "tests/test-images/wedding.jpg"
 TEST_FILE_BAD_IMAGE = "tests/test-images/badimage.jpeg"
@@ -89,6 +92,20 @@ EXIF_UUID_NO_GROUPS = {
 }
 EXIF_UUID_NONE = ["A1DD1F98-2ECD-431F-9AC9-5AFEFE2D3A5C"]
 
+QUOTED_JSON_BYTES = b'[{"ExifTool:ExifToolVersion": 12.37,"ExifTool:Now": "2022:02:22 18:14:31+00:00","ExifTool:NewGUID": "20220222181431005A76C1A4B4D508A2","ExifTool:FileSequence": 0,"ExifTool:Warning": "Error running "xattr" to extract XAttr tags","ExifTool:ProcessingTime": 0.157028}]'
+QUOTED_JSON_STRING_UNESCAPED = '[{"ExifTool:ExifToolVersion": 12.37,"ExifTool:Now": "2022:02:22 18:14:31+00:00","ExifTool:NewGUID": "20220222181431005A76C1A4B4D508A2","ExifTool:FileSequence": 0,"ExifTool:Warning": "Error running \\"xattr\\" to extract XAttr tags","ExifTool:ProcessingTime": 0.157028}]'
+QUOTED_JSON_LOADED = [
+    {
+        "ExifTool:ExifToolVersion": 12.37,
+        "ExifTool:Now": "2022:02:22 18:14:31+00:00",
+        "ExifTool:NewGUID": "20220222181431005A76C1A4B4D508A2",
+        "ExifTool:FileSequence": 0,
+        "ExifTool:Warning": 'Error running "xattr" to extract XAttr tags',
+        "ExifTool:ProcessingTime": 0.157028,
+    }
+]
+
+
 try:
     exiftool = get_exiftool_path()
 except:
@@ -126,6 +143,7 @@ def test_setvalue_1():
     # test setting a tag value
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -145,6 +163,7 @@ def test_setvalue_multiline():
     # test setting a tag value with embedded newline
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -164,6 +183,7 @@ def test_setvalue_non_alphanumeric_chars():
     # test setting a tag value non-alphanumeric characters
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -183,6 +203,7 @@ def test_setvalue_warning():
     # test setting illegal tag value generates warning
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -199,6 +220,7 @@ def test_setvalue_error():
     # test setting tag on bad image generates error
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -215,6 +237,7 @@ def test_setvalue_context_manager():
     # test setting a tag value as context manager
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -241,6 +264,7 @@ def test_setvalue_context_manager_warning():
     # test setting a tag value as context manager when warning generated
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -257,6 +281,7 @@ def test_setvalue_context_manager_error():
     # test setting a tag value as context manager when error generated
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -273,6 +298,7 @@ def test_flags():
     # test that flags work
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -296,6 +322,7 @@ def test_clear_value():
     # test clearing a tag value
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -315,6 +342,7 @@ def test_addvalues_1():
     # test setting a tag value
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -332,6 +360,7 @@ def test_addvalues_2():
     # test setting a tag value where multiple values already exist
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -353,6 +382,7 @@ def test_addvalues_non_alphanumeric_multiline():
     # test setting a tag value
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -373,6 +403,7 @@ def test_addvalues_unicode():
     # test setting a tag value with unicode
     import os.path
     import tempfile
+
     import osxphotos.exiftool
     from osxphotos.fileutil import FileUtil
 
@@ -444,9 +475,10 @@ def test_as_dict_no_tag_groups():
 
 
 def test_json():
-    import osxphotos.exiftool
     import json
 
+    import osxphotos.exiftool
+
     exif1 = osxphotos.exiftool.ExifTool(TEST_FILE_ONE_KEYWORD)
     exifdata = json.loads(exif1.json())
     assert exifdata[0]["XMP:TagsList"] == "wedding"
@@ -498,9 +530,10 @@ def test_photoinfo_exiftool_none():
 
 def test_exiftool_terminate():
     """Test that exiftool process is terminated when exiftool.terminate() is called"""
-    import osxphotos.exiftool
     import subprocess
 
+    import osxphotos.exiftool
+
     exif1 = osxphotos.exiftool.ExifTool(TEST_FILE_ONE_KEYWORD)
 
     ps = subprocess.run(["ps"], capture_output=True)
@@ -516,3 +549,11 @@ def test_exiftool_terminate():
     # verify we can create a new instance after termination
     exif2 = osxphotos.exiftool.ExifTool(TEST_FILE_ONE_KEYWORD)
     assert exif2.asdict()["IPTC:Keywords"] == "wedding"
+
+
+def test_unescape_str():
+    """Test unescape_str, #636"""
+    quoted_str = unescape_str(QUOTED_JSON_BYTES.decode("utf-8"))
+    assert quoted_str == QUOTED_JSON_STRING_UNESCAPED
+    quoted_json = json.loads(quoted_str)
+    assert quoted_json == QUOTED_JSON_LOADED

tests/test_export_db.py

@@ -177,6 +177,69 @@ def test_export_db_in_memory():
     assert uuids == [uuid]
 
 
+def test_export_db_in_memory_write_to_disk():
+    """test ExportDBInMemory with write back to disk"""
+    tempdir = tempfile.TemporaryDirectory(prefix="osxphotos_")
+    dbname = os.path.join(tempdir.name, ".osxphotos_export.db")
+    db = ExportDB(dbname, tempdir.name)
+    assert os.path.isfile(dbname)
+
+    filepath = os.path.join(tempdir.name, "test.JPG")
+
+    uuid = "FOOBAR"
+    record = db.create_file_record(filepath, uuid)
+    record.photoinfo = INFO_DATA
+    record.exifdata = EXIF_DATA
+    record.digest = DIGEST_DATA
+    record.src_sig = (7, 8, 9)
+    record.dest_sig = (10, 11, 12)
+    db.close()
+
+    # create in memory version
+    dbram = ExportDBInMemory(dbname, tempdir.name)
+    record2 = dbram.get_file_record(filepath)
+    assert record2.uuid == uuid
+    assert record2.photoinfo == INFO_DATA
+    assert record2.exifdata == EXIF_DATA
+    assert record2.digest == DIGEST_DATA
+    assert record2.src_sig == (7, 8, 9)
+    assert record2.dest_sig == (10, 11, 12)
+
+    # change some values
+    record2.photoinfo = INFO_DATA2
+    record2.exifdata = EXIF_DATA2
+    record2.digest = DIGEST_DATA2
+    record2.src_sig = (13, 14, 15)
+    record2.dest_sig = (16, 17, 18)
+
+    assert record2.photoinfo == INFO_DATA2
+    assert record2.exifdata == EXIF_DATA2
+    assert record2.digest == DIGEST_DATA2
+    assert record2.src_sig == (13, 14, 15)
+    assert record2.dest_sig == (16, 17, 18)
+
+    # all uuids
+    uuids = dbram.get_previous_uuids()
+    assert uuids == [uuid]
+
+    # write to disk
+    dbram.write_to_disk()
+    dbram.close()
+
+    # re-open original, assert changes are written back
+    db = ExportDB(dbname, tempdir.name)
+    record = db.get_file_record(filepath)
+    assert record.photoinfo == INFO_DATA2
+    assert record.exifdata == EXIF_DATA2
+    assert record.digest == DIGEST_DATA2
+    assert record.src_sig == (13, 14, 15)
+    assert record.dest_sig == (16, 17, 18)
+
+    # all uuids
+    uuids = db.get_previous_uuids()
+    assert uuids == [uuid]
+
+
 def test_export_db_in_memory_nofile():
     """test ExportDBInMemory with no dbfile"""
     tempdir = tempfile.TemporaryDirectory(prefix="osxphotos_")

tests/test_monterey_dev_beta_12_0_0.py

@@ -1356,7 +1356,7 @@ def test_exiftool_newlines_in_description(photosdb):
     assert exif["EXIF:ImageDescription"].find("\n") > 0
 
 
-@pytest.mark.skip(SKIP_TEST, reason="Not yet implemented")
+@pytest.mark.skip(reason="Test not yet implemented")
 def test_duplicates_1(photosdb):
     # test photo has duplicates
 

utils/update_readme.py

@@ -14,7 +14,7 @@ import re
 
 from click.testing import CliRunner
 
-from osxphotos.cli import help
+from osxphotos.cli import cli_main
 from osxphotos.phototemplate import (
     FILTER_VALUES,
     TEMPLATE_SUBSTITUTIONS,
@@ -55,7 +55,7 @@ TEMPLATE_SYSTEM_LINK_STOP = "<!-- OSXPHOTOS-TEMPLATE-SYSTEM-LINK:END -->"
 
 
 def generate_template_table():
-    """ generate template substitution table for README.md """
+    """generate template substitution table for README.md"""
 
     template_table = "| Substitution | Description |"
     template_table += "\n|--------------|-------------|"
@@ -68,21 +68,21 @@ def generate_template_table():
 
 
 def generate_help_text(command):
-    """ generate output of `osxphotos help command` """
+    """generate output of `osxphotos help command`"""
     runner = CliRunner()
 
     # get current help text
     with runner.isolated_filesystem():
-        result = runner.invoke(help, [command])
+        result = runner.invoke(cli_main, ["help", command])
         help_txt = result.output
 
     # running the help command above doesn't output the full "Usage" line
-    help_txt = help_txt.replace(f"Usage: {command}", f"Usage: osxphotos {command}")
+    help_txt = help_txt.replace(f"Usage: cli-main", f"Usage: osxphotos")
     return help_txt
 
 
 def replace_text(text, start_tag, stop_tag, replacement_text, prefix="", postfix=""):
-    """ replace text between start/stop tags with new text
+    """replace text between start/stop tags with new text
 
     Args:
         text: str, original text
@@ -108,13 +108,13 @@ def replace_text(text, start_tag, stop_tag, replacement_text, prefix="", postfix
         end = text.split(stop_tag)[1]
     except IndexError as e:
         # didn't find one of the delimiters
-        raise ValueError(f"Unable to parse input: {e}")
+        raise ValueError(f"Unable to parse input: {e}") from e
 
     return begin + start_tag + prefix + replacement_text + postfix + stop_tag + end
 
 
 def main():
-    """ update README.md """
+    """update README.md"""
     # update phototemplate.md with info on filters
     print(f"Updating {TEMPLATE_HELP}")
     filter_help = "\n".join(f"- {f}: {descr}" for f, descr in FILTER_VALUES.items())