Now copy write-ahead log and shared memory files which fixes problem with access to recently changed data

README.md

@@ -13,7 +13,7 @@ NOTE: OSXPhotos currently only supports image files -- e.g. it does not handle m
 
 Only works on MacOS (aka Mac OS X). Tested on MacOS 10.12.6 / Photos 2.0, 10.13.6 / Photos 3.0 and MacOS 10.14.5, 10.14.6 / Photos 4.0. Requires python >= 3.6
 
-NOTE: Alpha support for Mac OS 10.15.0 / Photos 5.0.  Photos 5.0 uses a new database format which required rewrite of much of the code for this package.  If you find bugs, please open an [issue](https://github.com/RhetTbull/osxphotos/issues/).
+NOTE: Alpha support for Mac OS 10.15.0 / Photos 5.0.  Photos 5.0 uses a new database format which required rewrite of much of the code for this module.  If you find bugs, please open an [issue](https://github.com/RhetTbull/osxphotos/issues/).
 
 This module will read Photos databases for any supported version on any supported OS version.  E.g. you can read a database created with Photos 4.0 on MacOS 10.14 on a machine running MacOS 10.12
 
@@ -88,7 +88,7 @@ Example: find all photos with keyword "Kids" and output results to json file nam
 
 `osxphotos query --keyword Kids --json >results.json`
 
-## Example uses of the module
+## Example uses of the module 
 
 ```python
 import osxphotos
@@ -377,11 +377,11 @@ This project started as a command line utility, `photosmeta`, available at [phot
 
 ## Implementation Notes
 
-This module is very kludgy.  It works by creating a copy of the sqlite3 database that Photos uses to store data about the Photos library. The class PhotosDB then queries this database to extract information about the photos such as persons (faces identified in the photos), albums, keywords, etc.  
+This module works by creating a copy of the sqlite3 database that Photos uses to store data about the Photos library. The class PhotosDB then queries this database to extract information about the photos such as persons (faces identified in the photos), albums, keywords, etc.  
 
-If Apple changes the database format this will likely break.  
+If Apple changes the database format this will likely break.
 
-The sqlite3 database used by Photos uses write ahead logging that is updated asynchronously in the background by a Photos helper service.  Sometimes the update takes a long time meaning the latest changes made in Photos (e.g. add a keyword) will not show up in the database for sometime.  I know of no way around this.
+Apple does provide an framework ([PhotoKit](https://developer.apple.com/documentation/photokit?language=objc)) for querying the user's Photos library and I attempted to create the funcationality in this module using this framework but unfortunately PhotoKit does not provide access to much of the needed metadata (such as Faces/Persons).  While copying the sqlite file is a bit kludgy, it allows osxphotos to provide access to all available metadata.
 
 ## Dependencies
 - [PyObjC](https://pythonhosted.org/pyobjc/)

osxphotos/__init__.py

@@ -21,7 +21,6 @@ from Foundation import *
 from . import _applescript
 
 # TODO: add hasAdjustments to process_database5 (see ZGENERICASSET.ZHASADJUSTMENTS = 1 )
-# TODO: Test to see if I really need to copy the database file
 # TODO: find edited photos: see https://github.com/orangeturtle739/photos-export/blob/master/extract_photos.py
 # TODO: Add test for imageTimeZoneOffsetSeconds = None
 # TODO: Fix command line so multiple --keyword, etc. are AND (instead of OR as they are in .photos())
@@ -322,19 +321,29 @@ class PhotosDB:
             print("Could not get path to Photos database", file=sys.stderr)
             return None
 
-    # TODO: do we need to copy the db-wal write-ahead log file?
     def _copy_db_file(self, fname):
         """ copies the sqlite database file to a temp file """
         """ returns the name of the temp file and appends name to self->_tmp_files """
+        """ If sqlite shared memory and write-ahead log files exist, those are copied too """
         # required because python's sqlite3 implementation can't read a locked file
-        _, tmp = tempfile.mkstemp(suffix=".db", prefix="photos")
+        _, suffix = os.path.splitext(fname)
+        tmp_files = []
         try:
+            _, tmp = tempfile.mkstemp(suffix=suffix, prefix="osxphotos-")
             copyfile(fname, tmp)
+            tmp_files.append(tmp)
+            # copy write-ahead log and shared memory files (-wal and -shm) files if they exist
+            if os.path.exists(f"{fname}-wal"):
+                copyfile(f"{fname}-wal", f"{tmp}-wal")
+                tmp_files.append(f"{tmp}-wal")
+            if os.path.exists(f"{fname}-shm"):
+                copyfile(f"{fname}-shm", f"{tmp}-shm")
+                tmp_files.append(f"{tmp}-shm")
         except:
             print("Error copying " + fname + " to " + tmp, file=sys.stderr)
             raise Exception
 
-        self._tmp_files.append(tmp)
+        self._tmp_files.extend(tmp_files)
         return tmp
 
     def _open_sql_file(self, file):

setup.py

@@ -38,7 +38,7 @@ with open(path.join(this_directory, "README.md"), encoding="utf-8") as f:
 
 setup(
     name="osxphotos",
-    version="0.14.5",
+    version="0.14.6",
     description="Manipulate (read-only) Apple's Photos app library on Mac OS X",
     long_description=long_description,
     long_description_content_type="text/markdown",