[request] export flags

[rb0022]

Export flag to control edited images / duplication when AAE file exists for image.

The AAE file is automatically created by iOS Photo app during edits (sidebar: or in my case, seemingly random image resizes - probably created automatically when sharing/sending the image)

At present, when performing extract via macOS Image Capture app 2 files are provided - the original and the AAE file that stores the edits. When extracting via finder + ios_backup_extractor, there is no AAE file extracted (this is good, as it cannot be used by anything except Apple software) and all edit/resize images are extracted too.

This can lead to significant junk extracted. In my test case:

• image capture app
  • 390 AAE
  • 390 JPG
• finder + ios_backup_extractor
  • 94 JPG
  • 182 JPG, where 91 unique images were extracted x2
  • 615 JPG where 205 unique images were extracted x3

== 891 files extracted, when it should be 390 files

image capture app

2015-03/
IMG_1636.AAE
IMG_E1636.JPG

IMG_1637.AAE
IMG_E1637.JPG

finder + ios_backup_extractor

2015-03/
IMG_1636.jpg    [original]
IMG_1636.0.jpg
IMG_1636.1.jpg

IMG_1637.0.jpg  [original]
IMG_1637.1.jpg
IMG_1637.2.jpg

---

Edit: This can be combined with other export flags to control extract of Instagram/WhatsApp and Screenshots into a separate extract subdirectory (i.e. Album) or images which are screenshots or selfies (i.e. Media Type).

Many of the Instagram or Screenshots would be unwanted in a personal photo collection on a backup HDD. See list of default tiles on iPhone....

• Album "Camera Roll"
• Album "Favorites"
• Album "Instagram"
• Album "WhatsApp"
• Media Type - Videos
• Media Type - Selfies
• Media Type - Panoramas
• Media Type - Bursts
• Media Type - Screenshots
• "Memories and Featured Photos"

[joz-k]

The idea of exporting all edits as separate files was sort of a design decision. As you correctly mentioned, the AAE metadata storage file is completely Apple-specific. Also, no duplicate (as in truly identical) files are ever extracted. A SHA-256 checksum is calculated for each file, and if the completely binary identical file is already exported, then the same file is never duplicated as another IMG_1637.X.jpg.

Anyway, do you have any idea how to improve this? I understand that many seemingly identical files can be exported this way. (Even though they are not binary identical.)

[rb0022]

@joz-k A sensible design decision, as some alterations to color or filters may be desired. However in my situation it was just bloat of the same image in a different (low) resolution.

To consolidate operator/logic suggestions here (instead of putting into each issue)...

1. Rename --format to -fd / --format-directory-structure [ym, ymd, flat]
2. Merge --prepend-date + --prepend-date-separator functionality and rename to -ff / --format-file-date [disabled, none, dash, underscore]
3. Create -ef / --export-flags [x,y,z]
   • sub_dir_edited_files which is used to alter current design, by putting original file in target path (e.g. ./[ym]/) and the edited files in target path + subdirectory (e.g. ./[ym]/edited)
   • parent_dir_original_name to create extra layer (e.g. ./101APPLE/[ym]/)
   • parent_dir_album_name to create extra layer (e.g. ./album_camera_roll/[ym]/)
   • parent_dir_media_type to create extra layer (e.g. ./type_selfies/[ym]/)

Although stacking those export flags would be difficult to code....

e.g. ./101APPLE/album_camera_roll/type_selfies/[ym]/edited

[joz-k]

@rb0022 Thanks for suggestions.

Items 1 and 2 are quite orthogonal to the problem of duplicated edited files. They're not bad suggestions, but I'm not sure the benefit would outweigh the incompatibility with previous versions.

I could add the flag to export edited files to a subdirectory (named after an original photo). However, this approach is not without new problems. For example, if the original photo is already deleted on the device (a common case, I think), you will only get a subdirectory of edited files, but not the original.

[rb0022]

@joz-k

Re. 1/2, I agree a change to operator names causes temporary pain for sanity in the long term. Formatting the directory structure and naming are separate to the file naming, by prefixing with --format-[directory,file]* it would hopefully be clearer for end-users and easier to extend with new operators.

Re. incompatibility from 1/2, if you wanted to be very kind I would replace the operator names and create 1.3.0 and leave the old operators hidden + working until 1.4.0; personally I would not be so kind as it would be quickly apparent and the end-user could check the README. The ~1000 downloads probably does not warrant being this kind (see https://githubstats0.firebaseapp.com) imho

Re. original photo deleted use case - I agree, hence the end-user would have to make the choice which --export-flags to be used. If the end-user chooses sub_dir_edited_files flag, they should expect that the edited photo they see on the device would be under the subdir.

I guess my overall perspective is:

• provide easy defaults like ym for directory name and disabled for file name prefix
• allow end-user to tweak the export directory structure, directory names and file names to their preferences

[rb0022]

@joz-k Can we please return to the above suggestions?

Improve operator names for long-term user clarity

This is a sidebar, we can create another GH Issue for tracking this

1. -fd / --format-directory-structure [ym (default) / ymd / flat]
2. -ff / --format-file-date [disabled (default) / none / dash / underscore]

Append export flags

Main objective of this GH Issue

3. -ef / --export-flags [optional,flags,separated]
   • sub_dir_edited_files separate edited files into subdirectory (e.g. ./[ym]/edited)
   • parent_dir_original_name add technical location to path (e.g. ./101APPLE/[ym]/)
   • parent_dir_album_name add album to path e.g. ./album_camera_roll/[ym]/)
   • parent_dir_media_type add medium type to path (e.g. ./type_selfies/[ym]/)
   • ignore_icloud_media to skip media on device downloaded from iCloud
   • ignore_social_media to skip media on device downloaded by WhatsApp or Instagram
   • ignore_screenshots to skip all screenshots
   • ignore_backup_version_check to skip version check of iOS backup files

When all export flags enabled, exports would use ./101APPLE/album_camera_roll/type_selfies/[ym]/edited and would only extract media that was not a screenshot and not from iCloud or Social Media (WhatsApp/Instagram)

[rb0022]

Appended to list, the flag mentioned in #17

[rb0022]

@joz-k can we please continue the above discussion?

I appreciate your preference to have --operator-names-that-are-easy value_goes_here or --simple-boolean-turn-on-operator-names, but as above shows (and other GH Issues) there are too many choices for how the files can be exported. The current operator naming is too restrictive for long-term.

I also appreciate, the above request is a breaking change that would create a v2.0.0 release.

[rb0022]

@joz-k any time to continue this discussion?

Really I would love to just add a PR myself to the project..... but Perl is a lang I'm honestly just not willing to learn.

[joz-k]

@rb0022 : Sorry for the delayed response to this request.

So far, I have understood this issue as an open discussion about changes to the export flags format. I have not yet made a commitment to implement any of the ideas we discussed.

The reason is that

• I haven't added a new feature that would warrant any changes to the export argument format in the last few months.
• Second, I'm still not sure about the -ef / --export-flags [optional,flags,separated] format. It is certainly more flexible but also more complex. I don't think this tool needs such an advanced export flag format yet. Perhaps there is no need to implement it yet.

[rb0022]

@joz-k Unless that is the solution? --expert-export-flags [optional,flags,separated] for an "expert mode" that overrides the regular functionality?

This would allow existing functionality to remain, while extending for those users who want more fine-tuning (see above suggested flags, which I would probably use all of them)