Extended attribute status already compressed files

[denji]

I had an idea to save extended attributes files on the file system so that when the next task on a large directory is recognized from files that have already been optimized, the attribute can store a hash or bitmask of the metadata file attributes of the leading optimization pass of the image_optim utility.

# Linux
getfattr, setfattr
# *BSD
getextattr, lsextattr, rmextattr, setextattr
# macOS
xattr

[kaspergrubbe]

That is an interesting idea, and I think ImageOptim.app for OSX does something like that, at least it seems to know which files it have already been optimising.

The stored hash should include data such as ImageOptim version, configuration such as which processors were used for optimising the image, and which settings those processors were used with, so if you update ImageOptim or change the relevant configuration it should cache-bust and optimise the image again.

Currently I store all originals, so that if ImageOptim destroys an image, or does something non-reversible to it, I can always go back in time, but it is very costly to reprocess them all.

[denji]

@kaspergrubbe It looks like ImageOptim uses MD5 for caching state and invalidate when changing the optimization settings:

• https://github.com/ImageOptim/ImageOptim/blob/ef44a49/imageoptim/Backend/Job.m#L643
• https://github.com/ImageOptim/ImageOptim/blob/ef44a49/imageoptim/Backend/Job.m#L321-L434

ImageOptim also support delete:

• Extended attribute{com.apple.FinderInfo, com.apple.ResourceFork, com.apple.quarantine, com.apple.metadata:kMDItemWhereFroms}
• EXIF.

[kaspergrubbe]

Nice digging, with code-examples and everything! I am not sure what you mean by "ImageOptim also support delete:".

[toy]

If enabling caching is an option, it will make image_optim not try to optimise files second time.

I really don't know if extended attributes are often enabled, on mac os x they are, but I had a problem with linux machines on travis as they did not have it enabled when I wanted to test my other gem fspath-xattr. Anyway removing those special extended attributes on osx can be an option for image_optim, it will reduce overall file size. EXIF is already handled by multiple tools.

If I understand correctly, ImageOptim saves information about used workers in sqlite db.