Removed debug code from exiftool, fixed #641

Updated comment for #636
pass PATH to exiftool to find xattr
2022-02-24 05:09:42 -08:00 · 2022-02-23 06:15:03 -08:00 · 2022-02-22 22:11:12 -08:00 · 2022-02-22 09:49:02 -08:00 · 2022-02-22 06:40:25 -08:00 · 2022-02-21 22:58:54 -08:00
22 changed files with 229 additions and 86 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. Dates are d

 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).

+#### [v0.46.1](https://github.com/RhetTbull/osxphotos/compare/v0.46.0...v0.46.1)
+
+> 21 February 2022
+
+- Added --ramdb option [`#639`](https://github.com/RhetTbull/osxphotos/pull/639)
+
 #### [v0.46.0](https://github.com/RhetTbull/osxphotos/compare/v0.45.12...v0.46.0)

 > 21 February 2022
--- a/README.md
+++ b/README.md
@@ -1741,7 +1741,7 @@ Substitution                    Description
 {lf}                            A line feed: '\n', alias for {newline}
 {cr}                            A carriage return: '\r'
 {crlf}                          a carriage return + line feed: '\r\n'
-{osxphotos_version}             The osxphotos version, e.g. '0.46.1'
+{osxphotos_version}             The osxphotos version, e.g. '0.46.4'
 {osxphotos_cmd_line}            The full command line used to run osxphotos

 The following substitutions may result in multiple values. Thus if specified for
@@ -3645,7 +3645,7 @@ The following template field substitutions are availabe for use the templating s
 |{lf}|A line feed: '\n', alias for {newline}|
 |{cr}|A carriage return: '\r'|
 |{crlf}|a carriage return + line feed: '\r\n'|
-|{osxphotos_version}|The osxphotos version, e.g. '0.46.1'|
+|{osxphotos_version}|The osxphotos version, e.g. '0.46.4'|
 |{osxphotos_cmd_line}|The full command line used to run osxphotos|
 |{album}|Album(s) photo is contained in|
 |{folder_album}|Folder path + album photo is contained in. e.g. 'Folder/Subfolder/Album' or just 'Album' if no enclosing folder|
--- a/docs/.buildinfo
+++ b/docs/.buildinfo
@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: d6da9902a4771e5081ae73c361960af8
+config: 4fd4a10e261cc9bab3c5f7edf97d5f38
 tags: 645f666f9bcd5a90fca523b33c5a78b7
--- a/docs/_modules/index.html
+++ b/docs/_modules/index.html
@@ -5,7 +5,7 @@
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Overview: module code &#8212; osxphotos 0.46.1 documentation</title>
+    <title>Overview: module code &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="../_static/alabaster.css" />
    <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
--- a/docs/_modules/osxphotos/photoinfo.html
+++ b/docs/_modules/osxphotos/photoinfo.html
@@ -5,7 +5,7 @@
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>osxphotos.photoinfo &#8212; osxphotos 0.46.1 documentation</title>
+    <title>osxphotos.photoinfo &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="../../_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="../../_static/alabaster.css" />
    <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
@@ -87,7 +87,7 @@
 <span class="kn">from</span> <span class="nn">.searchinfo</span> <span class="kn">import</span> <span class="n">SearchInfo</span>
 <span class="kn">from</span> <span class="nn">.text_detection</span> <span class="kn">import</span> <span class="n">detect_text</span>
 <span class="kn">from</span> <span class="nn">.uti</span> <span class="kn">import</span> <span class="n">get_preferred_uti_extension</span><span class="p">,</span> <span class="n">get_uti_for_extension</span>
-<span class="kn">from</span> <span class="nn">.utils</span> <span class="kn">import</span> <span class="n">_debug</span><span class="p">,</span> <span class="n">_get_resource_loc</span><span class="p">,</span> <span class="n">list_directory</span>
+<span class="kn">from</span> <span class="nn">.utils</span> <span class="kn">import</span> <span class="n">_debug</span><span class="p">,</span> <span class="n">_get_resource_loc</span><span class="p">,</span> <span class="n">list_directory</span><span class="p">,</span> <span class="n">_debug</span>

 <span class="n">__all__</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;PhotoInfo&quot;</span><span class="p">,</span> <span class="s2">&quot;PhotoInfoNone&quot;</span><span class="p">]</span>

@@ -621,7 +621,7 @@
    <span class="nd">@property</span>
    <span class="k">def</span> <span class="nf">ismissing</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="sd">&quot;&quot;&quot;returns true if photo is missing from disk (which means it&#39;s not been downloaded from iCloud)</span>
-<span class="sd">        </span>
+
 <span class="sd">        NOTE:   the photos.db database uses an asynchrounous write-ahead log so changes in Photos</span>
 <span class="sd">                do not immediately get written to disk. In particular, I&#39;ve noticed that downloading</span>
 <span class="sd">                an image from the cloud does not force the database to be updated until something else</span>
--- a/docs/_modules/osxphotos/photosdb/photosdb.html
+++ b/docs/_modules/osxphotos/photosdb/photosdb.html
@@ -5,7 +5,7 @@
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>osxphotos.photosdb.photosdb &#8212; osxphotos 0.46.0 documentation</title>
+    <title>osxphotos.photosdb.photosdb &#8212; osxphotos 0.46.2 documentation</title>
    <link rel="stylesheet" type="text/css" href="../../../_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="../../../_static/alabaster.css" />
    <script data-url_root="../../../" id="documentation_options" src="../../../_static/documentation_options.js"></script>
@@ -3312,27 +3312,6 @@
        <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">to_time</span><span class="p">:</span>
            <span class="n">photos</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span> <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">&lt;=</span> <span class="n">options</span><span class="o">.</span><span class="n">to_time</span><span class="p">]</span>

-        <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">:</span>
-            <span class="c1"># add the burst_photos to the export set</span>
-            <span class="n">photos_burst</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span> <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">burst</span><span class="p">]</span>
-            <span class="k">for</span> <span class="n">burst</span> <span class="ow">in</span> <span class="n">photos_burst</span><span class="p">:</span>
-                <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">missing_bursts</span><span class="p">:</span>
-                    <span class="c1"># include burst photos that are missing</span>
-                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">)</span>
-                <span class="k">else</span><span class="p">:</span>
-                    <span class="c1"># don&#39;t include missing burst images (these can&#39;t be downloaded with AppleScript)</span>
-                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">p</span><span class="o">.</span><span class="n">ismissing</span><span class="p">])</span>
-
-            <span class="c1"># remove duplicates as each burst photo in the set that&#39;s selected would</span>
-            <span class="c1"># result in the entire set being added above</span>
-            <span class="c1"># can&#39;t use set() because PhotoInfo not hashable</span>
-            <span class="n">seen_uuids</span> <span class="o">=</span> <span class="p">{}</span>
-            <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span><span class="p">:</span>
-                <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">uuid</span> <span class="ow">in</span> <span class="n">seen_uuids</span><span class="p">:</span>
-                    <span class="k">continue</span>
-                <span class="n">seen_uuids</span><span class="p">[</span><span class="n">p</span><span class="o">.</span><span class="n">uuid</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span>
-            <span class="n">photos</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">seen_uuids</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>
-
        <span class="k">if</span> <span class="n">name</span><span class="p">:</span>
            <span class="c1"># search filename fields for text</span>
            <span class="c1"># if more than one, find photos with all title values in filename</span>
@@ -3483,6 +3462,28 @@
            <span class="k">for</span> <span class="n">function</span> <span class="ow">in</span> <span class="n">options</span><span class="o">.</span><span class="n">function</span><span class="p">:</span>
                <span class="n">photos</span> <span class="o">=</span> <span class="n">function</span><span class="p">[</span><span class="mi">0</span><span class="p">](</span><span class="n">photos</span><span class="p">)</span>

+        <span class="c1"># burst should be checked last, ref #640</span>
+        <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">:</span>
+            <span class="c1"># add the burst_photos to the export set</span>
+            <span class="n">photos_burst</span> <span class="o">=</span> <span class="p">[</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span> <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">burst</span><span class="p">]</span>
+            <span class="k">for</span> <span class="n">burst</span> <span class="ow">in</span> <span class="n">photos_burst</span><span class="p">:</span>
+                <span class="k">if</span> <span class="n">options</span><span class="o">.</span><span class="n">missing_bursts</span><span class="p">:</span>
+                    <span class="c1"># include burst photos that are missing</span>
+                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span><span class="p">)</span>
+                <span class="k">else</span><span class="p">:</span>
+                    <span class="c1"># don&#39;t include missing burst images (these can&#39;t be downloaded with AppleScript)</span>
+                    <span class="n">photos</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span><span class="n">p</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">burst</span><span class="o">.</span><span class="n">burst_photos</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">p</span><span class="o">.</span><span class="n">ismissing</span><span class="p">])</span>
+
+            <span class="c1"># remove duplicates as each burst photo in the set that&#39;s selected would</span>
+            <span class="c1"># result in the entire set being added above</span>
+            <span class="c1"># can&#39;t use set() because PhotoInfo not hashable</span>
+            <span class="n">seen_uuids</span> <span class="o">=</span> <span class="p">{}</span>
+            <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">photos</span><span class="p">:</span>
+                <span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">uuid</span> <span class="ow">in</span> <span class="n">seen_uuids</span><span class="p">:</span>
+                    <span class="k">continue</span>
+                <span class="n">seen_uuids</span><span class="p">[</span><span class="n">p</span><span class="o">.</span><span class="n">uuid</span><span class="p">]</span> <span class="o">=</span> <span class="n">p</span>
+            <span class="n">photos</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">seen_uuids</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>
+
        <span class="k">return</span> <span class="n">photos</span></div>

 <div class="viewcode-block" id="PhotosDB.execute"><a class="viewcode-back" href="../../../reference.html#osxphotos.PhotosDB.execute">[docs]</a>    <span class="k">def</span> <span class="nf">execute</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sql</span><span class="p">):</span>
@@ -3568,7 +3569,6 @@
 <h3>Navigation</h3>
 <ul>
 <li class="toctree-l1"><a class="reference internal" href="../../../cli.html">osxphotos command line interface (CLI)</a></li>
-<li class="toctree-l1"><a class="reference internal" href="../../../modules.html">osxphotos</a></li>
 <li class="toctree-l1"><a class="reference internal" href="../../../reference.html">osxphotos package</a></li>
 </ul>

--- a/docs/_static/documentation_options.js
+++ b/docs/_static/documentation_options.js
@@ -1,6 +1,6 @@
 var DOCUMENTATION_OPTIONS = {
    URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
-    VERSION: '0.46.1',
+    VERSION: '0.46.4',
    LANGUAGE: 'None',
    COLLAPSE_INDEX: false,
    BUILDER: 'html',
--- a/docs/cli.html
+++ b/docs/cli.html
@@ -6,7 +6,7 @@
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />

-    <title>osxphotos command line interface (CLI) &#8212; osxphotos 0.46.1 documentation</title>
+    <title>osxphotos command line interface (CLI) &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
--- a/docs/genindex.html
+++ b/docs/genindex.html
@@ -5,7 +5,7 @@
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Index &#8212; osxphotos 0.46.1 documentation</title>
+    <title>Index &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
--- a/docs/index.html
+++ b/docs/index.html
@@ -6,7 +6,7 @@
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />

-    <title>Welcome to osxphotos’s documentation! &#8212; osxphotos 0.46.1 documentation</title>
+    <title>Welcome to osxphotos’s documentation! &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
--- a/docs/modules.html
+++ b/docs/modules.html
@@ -6,7 +6,7 @@
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />

-    <title>osxphotos &#8212; osxphotos 0.46.1 documentation</title>
+    <title>osxphotos &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
--- a/docs/reference.html
+++ b/docs/reference.html
@@ -6,7 +6,7 @@
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />

-    <title>osxphotos package &#8212; osxphotos 0.46.1 documentation</title>
+    <title>osxphotos package &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
--- a/docs/search.html
+++ b/docs/search.html
@@ -5,7 +5,7 @@
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Search &#8212; osxphotos 0.46.1 documentation</title>
+    <title>Search &#8212; osxphotos 0.46.4 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
    
--- a/osxphotos/_version.py
+++ b/osxphotos/_version.py
@@ -1,3 +1,3 @@
 """ version info """

-__version__ = "0.46.1"
+__version__ = "0.46.4"
--- a/osxphotos/cli.py
+++ b/osxphotos/cli.py
@@ -2298,6 +2298,9 @@ def help(ctx, topic, **kw):
    "This only works if the Photos library being queried is the last-opened (default) library in Photos. "
    "This feature is currently experimental.  I don't know how well it will work on large query sets.",
 )
+@click.option(
+    "--debug", required=False, is_flag=True, default=False, hidden=OSXPHOTOS_HIDDEN
+)
@DB_ARGUMENT
@click.pass_obj
@click.pass_context
@@ -2382,12 +2385,18 @@ def query(
    query_eval,
    query_function,
    add_to_album,
+    debug,
 ):
    """Query the Photos database using 1 or more search options;
    if more than one option is provided, they are treated as "AND"
    (e.g. search for photos matching all options).
    """

+    global DEBUG
+    if debug:
+        DEBUG = True
+        osxphotos._set_debug(True)
+
    # if no query terms, show help and return
    # sanity check input args
    nonexclusive = [
@@ -4808,6 +4817,11 @@ def run(python_file):
    is_flag=True,
    help="Migrate (if needed) export database to current version.",
 )
+@click.option(
+    "--sql",
+    metavar="SQL_STATEMENT",
+    help="Execute SQL_STATEMENT against export database and print results.",
+)
@click.option(
    "--export-dir",
    help="Optional path to export directory (if not parent of export database).",
@@ -4830,6 +4844,7 @@ def exportdb(
    save_config,
    info,
    migrate,
+    sql,
    export_dir,
    verbose,
    dry_run,
@@ -4857,6 +4872,7 @@ def exportdb(
        bool(save_config),
        bool(info),
        migrate,
+        bool(sql),
    ]
    if sum(sub_commands) > 1:
        print(f"[red]Only a single sub-command may be specified at a time[/red]")
@@ -4975,6 +4991,19 @@ def exportdb(
            )
        sys.exit(0)

+    if sql:
+        exportdb = ExportDB(export_db, export_dir)
+        try:
+            c = exportdb._conn.cursor()
+            results = c.execute(sql)
+        except Exception as e:
+            print(f"[red]Error: {e}[/red]")
+            sys.exit(1)
+        else:
+            for row in results:
+                print(row)
+            sys.exit(0)
+

 def _query_options_from_kwargs(**kwargs) -> QueryOptions:
    """Validate query options and create a QueryOptions instance"""
--- a/osxphotos/exiftool.py
+++ b/osxphotos/exiftool.py
@@ -69,6 +69,8 @@ def unescape_str(s):
    """unescape an HTML string returned by exiftool -E"""
    if type(s) != str:
        return s
+    # avoid " in values which result in json.loads() throwing an exception, #636
+    s = s.replace("&quot;", '\\"')
    return html.unescape(s)


@@ -105,7 +107,8 @@ class _ExifToolProc:

    def __init__(self, exiftool=None):
        """construct _ExifToolProc singleton object or return instance of already created object
-        exiftool: optional path to exiftool binary (if not provided, will search path to find it)"""
+        exiftool: optional path to exiftool binary (if not provided, will search path to find it)
+        """

        if hasattr(self, "_process_running") and self._process_running:
            # already running
@@ -115,7 +118,6 @@ class _ExifToolProc:
                    f"ignoring exiftool={exiftool}"
                )
            return
-
        self._process_running = False
        self._exiftool = exiftool or get_exiftool_path()
        self._start_proc()
@@ -147,6 +149,9 @@ class _ExifToolProc:
            return

        # open exiftool process
+        # make sure /usr/bin at start of path so exiftool can find xattr (see #636)
+        env = os.environ.copy()
+        env["PATH"] = f'/usr/bin/:{env["PATH"]}'
        self._process = subprocess.Popen(
            [
                self._exiftool,
@@ -163,6 +168,7 @@ class _ExifToolProc:
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
+            env=env,
        )
        self._process_running = True

@@ -362,6 +368,7 @@ class ExifTool:
        error = "" if error == b"" else error.decode("utf-8")
        self.warning = warning
        self.error = error
+
        return output[:-EXIFTOOL_STAYOPEN_EOF_LEN], warning, error

    @property
@@ -393,6 +400,7 @@ class ExifTool:
        except Exception as e:
            # will fail with some commands, e.g --ext AVI which produces
            # 'No file with specified extension' instead of json
+            logging.warning(f"error loading json returned by exiftool: {e} {json_str}")
            return dict()
        exifdict = exifdict[0]
        if not tag_groups:
--- a/osxphotos/photoexporter.py
+++ b/osxphotos/photoexporter.py
@@ -560,11 +560,15 @@ class PhotoExporter:
        touch_results = []
        for touch_file in set(touch_files):
            ts = int(self.photo.date.timestamp())
-            stat = os.stat(touch_file)
-            if stat.st_mtime != ts:
-                if not options.dry_run:
+            try:
+                stat = os.stat(touch_file)
+                if stat.st_mtime != ts:
                    fileutil.utime(touch_file, (ts, ts))
-                touch_results.append(touch_file)
+                    touch_results.append(touch_file)
+            except FileNotFoundError as e:
+                # ignore errors if in dry_run as file may not be present
+                if not options.dry_run:
+                    raise e from e
        return ExportResults(touched=touch_results)

    def _get_edited_filename(self, original_filename):
@@ -669,8 +673,8 @@ class PhotoExporter:

        if file_record.export_options != options.bit_flags:
            # exporting with different set of options (e.g. exiftool), should update
-            # need to check this before exiftool in case exiftool options are different 
-            # and export database is missing; this will always be True if database is missing 
+            # need to check this before exiftool in case exiftool options are different
+            # and export database is missing; this will always be True if database is missing
            # as it'll be None and bit_flags will be an int
            return True

--- a/osxphotos/photoinfo.py
+++ b/osxphotos/photoinfo.py
@@ -54,7 +54,7 @@ from .scoreinfo import ScoreInfo
 from .searchinfo import SearchInfo
 from .text_detection import detect_text
 from .uti import get_preferred_uti_extension, get_uti_for_extension
-from .utils import _debug, _get_resource_loc, list_directory
+from .utils import _debug, _get_resource_loc, list_directory, _debug

 __all__ = ["PhotoInfo", "PhotoInfoNone"]

@@ -588,7 +588,7 @@ class PhotoInfo:
    @property
    def ismissing(self):
        """returns true if photo is missing from disk (which means it's not been downloaded from iCloud)
-        
+
        NOTE:   the photos.db database uses an asynchrounous write-ahead log so changes in Photos
                do not immediately get written to disk. In particular, I've noticed that downloading
                an image from the cloud does not force the database to be updated until something else
--- a/osxphotos/photosdb/photosdb.py
+++ b/osxphotos/photosdb/photosdb.py
@@ -3279,27 +3279,6 @@ class PhotosDB:
        if options.to_time:
            photos = [p for p in photos if p.date.time() <= options.to_time]

-        if options.burst_photos:
-            # add the burst_photos to the export set
-            photos_burst = [p for p in photos if p.burst]
-            for burst in photos_burst:
-                if options.missing_bursts:
-                    # include burst photos that are missing
-                    photos.extend(burst.burst_photos)
-                else:
-                    # don't include missing burst images (these can't be downloaded with AppleScript)
-                    photos.extend([p for p in burst.burst_photos if not p.ismissing])
-
-            # remove duplicates as each burst photo in the set that's selected would
-            # result in the entire set being added above
-            # can't use set() because PhotoInfo not hashable
-            seen_uuids = {}
-            for p in photos:
-                if p.uuid in seen_uuids:
-                    continue
-                seen_uuids[p.uuid] = p
-            photos = list(seen_uuids.values())
-
        if name:
            # search filename fields for text
            # if more than one, find photos with all title values in filename
@@ -3450,6 +3429,28 @@ class PhotosDB:
            for function in options.function:
                photos = function[0](photos)

+        # burst should be checked last, ref #640
+        if options.burst_photos:
+            # add the burst_photos to the export set
+            photos_burst = [p for p in photos if p.burst]
+            for burst in photos_burst:
+                if options.missing_bursts:
+                    # include burst photos that are missing
+                    photos.extend(burst.burst_photos)
+                else:
+                    # don't include missing burst images (these can't be downloaded with AppleScript)
+                    photos.extend([p for p in burst.burst_photos if not p.ismissing])
+
+            # remove duplicates as each burst photo in the set that's selected would
+            # result in the entire set being added above
+            # can't use set() because PhotoInfo not hashable
+            seen_uuids = {}
+            for p in photos:
+                if p.uuid in seen_uuids:
+                    continue
+                seen_uuids[p.uuid] = p
+            photos = list(seen_uuids.values())
+
        return photos

    def execute(self, sql):
--- a/tests/README.md
+++ b/tests/README.md
@@ -1,14 +1,18 @@
 # Tests for osxphotos #

 ## Running Tests ##
-Tests require pytest and pytest-mock:
-`pip install pytest`
-`pip install pytest-mock`
+To set up a dev environment to work on osxphotos code or run tests follow these steps.  This assumes you have python 3.7 or later installed.  If you need to install python, you can do so with the XCode command lines tools (`xcode-select --install`) or from [python.org](https://www.python.org/downloads/macos/).
+
+- `git clone git@github.com:RhetTbull/osxphotos.git`
+- `cd osxphotos`
+- `python3 -m venv venv`
+- `source venv/bin/activate`
+- `python3 -m pip install -r dev_requirements.txt`
+- `python3 -m pip install -e .`

 To run the tests, do the following from the main source folder:
-`python -m pytest tests/`
+`python3 -m pytest tests/`

-Running the tests this way allows the library to be tested without installing it.

 ## Skipped Tests ##
 A few tests will look for certain environment variables to determine if they should run.
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -6710,6 +6710,7 @@ def test_export_exportdb():
            in result.output
        )

+
 def test_export_exportdb_ramdb():
    """test --exportdb --ramdb"""
    import glob
@@ -6726,7 +6727,14 @@ def test_export_exportdb_ramdb():
    with runner.isolated_filesystem():
        result = runner.invoke(
            export,
-            [os.path.join(cwd, CLI_PHOTOS_DB), ".", "-V", "--exportdb", "export.db", "--ramdb"],
+            [
+                os.path.join(cwd, CLI_PHOTOS_DB),
+                ".",
+                "-V",
+                "--exportdb",
+                "export.db",
+                "--ramdb",
+            ],
        )
        assert result.exit_code == 0
        assert re.search(r"Created export database.*export\.db", result.output)
@@ -6742,7 +6750,7 @@ def test_export_exportdb_ramdb():
                "--exportdb",
                "export.db",
                "--update",
-                "--ramdb"
+                "--ramdb",
            ],
        )
        assert result.exit_code == 0
@@ -6773,13 +6781,7 @@ def test_export_ramdb():
        # run again, update should update no files if db written back to disk
        result = runner.invoke(
            export,
-            [
-                os.path.join(cwd, CLI_PHOTOS_DB),
-                ".",
-                "-V",
-                "--update",
-                "--ramdb"
-            ],
+            [os.path.join(cwd, CLI_PHOTOS_DB), ".", "-V", "--update", "--ramdb"],
        )
        assert result.exit_code == 0
        assert "exported: 0" in result.output
@@ -7413,6 +7415,54 @@ def test_export_burst_folder_album():
            assert sorted(files) == sorted(UUID_BURST_ALBUM[uuid])


+@pytest.mark.skipif(
+    "OSXPHOTOS_TEST_EXPORT" not in os.environ,
+    reason="Skip if not running on author's personal library.",
+)
+def test_export_burst_uuid():
+    """test non-selected burst photos are exported when image is specified by --uuid, #640"""
+    import glob
+    import os
+    import os.path
+    import pathlib
+
+    from osxphotos.cli import export
+
+    runner = CliRunner()
+    cwd = os.getcwd()
+    # pylint: disable=not-context-manager
+    for uuid in UUID_BURST_ALBUM:
+        with runner.isolated_filesystem():
+            result = runner.invoke(
+                export,
+                [
+                    os.path.join(cwd, PHOTOS_DB_RHET),
+                    ".",
+                    "-V",
+                    "--uuid",
+                    uuid,
+                ],
+            )
+            assert result.exit_code == 0
+            # subtract 1 from len because one photo in two albums so shows up twice in the list
+            assert f"exported: {len(UUID_BURST_ALBUM[uuid]) - 1}" in result.output
+
+            # export again with --skip-bursts
+            result = runner.invoke(
+                export,
+                [
+                    os.path.join(cwd, PHOTOS_DB_RHET),
+                    ".",
+                    "-V",
+                    "--uuid",
+                    uuid,
+                    "--skip-bursts",
+                ],
+            )
+            assert result.exit_code == 0
+            assert f"exported: 1" in result.output
+
+
@pytest.mark.skipif(
    "OSXPHOTOS_TEST_EXPORT" not in os.environ,
    reason="Skip if not running on author's personal library.",
--- a/tests/test_exiftool.py
+++ b/tests/test_exiftool.py
@@ -1,5 +1,8 @@
+import json
+
 import pytest
-from osxphotos.exiftool import get_exiftool_path
+
+from osxphotos.exiftool import get_exiftool_path, unescape_str

 TEST_FILE_ONE_KEYWORD = "tests/test-images/wedding.jpg"
 TEST_FILE_BAD_IMAGE = "tests/test-images/badimage.jpeg"
@@ -89,6 +92,20 @@ EXIF_UUID_NO_GROUPS = {
 }
 EXIF_UUID_NONE = ["A1DD1F98-2ECD-431F-9AC9-5AFEFE2D3A5C"]

+QUOTED_JSON_BYTES = b'[{"ExifTool:ExifToolVersion": 12.37,"ExifTool:Now": "2022:02:22 18:14:31+00:00","ExifTool:NewGUID": "20220222181431005A76C1A4B4D508A2","ExifTool:FileSequence": 0,"ExifTool:Warning": "Error running &quot;xattr&quot; to extract XAttr tags","ExifTool:ProcessingTime": 0.157028}]'
+QUOTED_JSON_STRING_UNESCAPED = '[{"ExifTool:ExifToolVersion": 12.37,"ExifTool:Now": "2022:02:22 18:14:31+00:00","ExifTool:NewGUID": "20220222181431005A76C1A4B4D508A2","ExifTool:FileSequence": 0,"ExifTool:Warning": "Error running \\"xattr\\" to extract XAttr tags","ExifTool:ProcessingTime": 0.157028}]'
+QUOTED_JSON_LOADED = [
+    {
+        "ExifTool:ExifToolVersion": 12.37,
+        "ExifTool:Now": "2022:02:22 18:14:31+00:00",
+        "ExifTool:NewGUID": "20220222181431005A76C1A4B4D508A2",
+        "ExifTool:FileSequence": 0,
+        "ExifTool:Warning": 'Error running "xattr" to extract XAttr tags',
+        "ExifTool:ProcessingTime": 0.157028,
+    }
+]
+
+
 try:
    exiftool = get_exiftool_path()
 except:
@@ -126,6 +143,7 @@ def test_setvalue_1():
    # test setting a tag value
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -145,6 +163,7 @@ def test_setvalue_multiline():
    # test setting a tag value with embedded newline
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -164,6 +183,7 @@ def test_setvalue_non_alphanumeric_chars():
    # test setting a tag value non-alphanumeric characters
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -183,6 +203,7 @@ def test_setvalue_warning():
    # test setting illegal tag value generates warning
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -199,6 +220,7 @@ def test_setvalue_error():
    # test setting tag on bad image generates error
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -215,6 +237,7 @@ def test_setvalue_context_manager():
    # test setting a tag value as context manager
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -241,6 +264,7 @@ def test_setvalue_context_manager_warning():
    # test setting a tag value as context manager when warning generated
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -257,6 +281,7 @@ def test_setvalue_context_manager_error():
    # test setting a tag value as context manager when error generated
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -273,6 +298,7 @@ def test_flags():
    # test that flags work
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -296,6 +322,7 @@ def test_clear_value():
    # test clearing a tag value
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -315,6 +342,7 @@ def test_addvalues_1():
    # test setting a tag value
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -332,6 +360,7 @@ def test_addvalues_2():
    # test setting a tag value where multiple values already exist
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -353,6 +382,7 @@ def test_addvalues_non_alphanumeric_multiline():
    # test setting a tag value
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -373,6 +403,7 @@ def test_addvalues_unicode():
    # test setting a tag value with unicode
    import os.path
    import tempfile
+
    import osxphotos.exiftool
    from osxphotos.fileutil import FileUtil

@@ -444,9 +475,10 @@ def test_as_dict_no_tag_groups():


 def test_json():
-    import osxphotos.exiftool
    import json

+    import osxphotos.exiftool
+
    exif1 = osxphotos.exiftool.ExifTool(TEST_FILE_ONE_KEYWORD)
    exifdata = json.loads(exif1.json())
    assert exifdata[0]["XMP:TagsList"] == "wedding"
@@ -498,9 +530,10 @@ def test_photoinfo_exiftool_none():

 def test_exiftool_terminate():
    """Test that exiftool process is terminated when exiftool.terminate() is called"""
-    import osxphotos.exiftool
    import subprocess

+    import osxphotos.exiftool
+
    exif1 = osxphotos.exiftool.ExifTool(TEST_FILE_ONE_KEYWORD)

    ps = subprocess.run(["ps"], capture_output=True)
@@ -516,3 +549,11 @@ def test_exiftool_terminate():
    # verify we can create a new instance after termination
    exif2 = osxphotos.exiftool.ExifTool(TEST_FILE_ONE_KEYWORD)
    assert exif2.asdict()["IPTC:Keywords"] == "wedding"
+
+
+def test_unescape_str():
+    """Test unescape_str, #636"""
+    quoted_str = unescape_str(QUOTED_JSON_BYTES.decode("utf-8"))
+    assert quoted_str == QUOTED_JSON_STRING_UNESCAPED
+    quoted_json = json.loads(quoted_str)
+    assert quoted_json == QUOTED_JSON_LOADED
Author	SHA1	Message	Date
Rhet Turnbull	173e3ccc37	Removed debug code from exiftool, fixed #641	2022-02-24 05:09:42 -08:00
Rhet Turnbull	9964fd0635	Updated comment for #636	2022-02-23 06:15:03 -08:00
Rhet Turnbull	e789cd5e9d	pass PATH to exiftool to find xattr	2022-02-22 22:11:12 -08:00
Rhet Turnbull	6cb7dedd9b	Updated debug info	2022-02-22 09:49:02 -08:00
Rhet Turnbull	39ba17dd1c	Added debug output to exiftool	2022-02-22 06:40:25 -08:00
Rhet Turnbull	5b66962ac1	Fixed export of bursts with --uuid and --selected, #640	2022-02-21 22:58:54 -08:00
Rhet Turnbull	c05340f631	removed macos-12 as it doesn't work with Actions	2022-02-21 22:49:12 -08:00
Rhet Turnbull	f24c461cbb	Added macos-12	2022-02-21 17:08:27 -08:00
Rhet Turnbull	c8ee679799	Added --sql command to exportdb	2022-02-21 16:06:16 -08:00
Rhet Turnbull	2966c9a60f	Updated docs [skip ci]	2022-02-21 15:17:19 -08:00
Rhet Turnbull	acfcb0c49a	Updated CHANGELOG.md [skip ci]	2022-02-21 11:39:04 -08:00