Adobe DNG: recognize EXIF tags (#114)

[twest820]

[drewnoakes]

This set of DNG tags looks great. They should live on a DNG-specific directory, as they're not part of the Exif spec. ExifDirectoryBase only contain tags found in the Exif spec, not necessarily the DNG spec. With the changes currently in this PR, a JPEG file might start showing DNG tags if the tag IDs collided, which could be confusing.

The challenge here is whether we can correctly identify DNG files and swap in the appropriate directory during TIFF processing.

[drewnoakes]

If the DNG spec is a super-set of the Exif tags, then a new DngDirectory could derive from ExifDirectoryBase.

[twest820]

DNG is an extension of TIFF 6.0. The present PR seems consistent with Metadata Extractor's current design as

• it lacks a TiffDirectory to derive from
• ExifIFD0Directory, ExifSubIfdDirectory, and several others call into ExifDirectoryBase for known tags

Results also appear consistent with presentation of DNG tags by other tools, such as exiv2. This leads me to wonder if the issue might be more usefully expressed as a failure to supply scope to known tag generation. Staleness and ongoing lack of CRLF canonicalization in metadata-extractor-images makes full checking impractical but I didn't notice any collisions in spot checks with FileProcessor.exe updated with this PR.

Curiously, the .dngs in metadata-extractor-images lack DNG tags and therefore don't seem representative of output from Adobe DNG Converter 10.1 (the current version). There are a few other sources, such as Pentax cameras, but DNG Converter is likely still a majority source of .dngs.

[paperboyo]

There are a few other sources, such as Pentax cameras, but DNG Converter is likely still a majority source of .dngs.

FYI, all Android phones capable of shooting RAW output DNG files. As do multiple camera apps for Android, iOS and Windows (Nokia). My totally unsubstantiated hunch is that DNG Converter isn’t a source of majority of DNGs in the wild any more.

[drewnoakes]

it lacks a TiffDirectory to derive from

What members would this class have? I can't see a need for it.

ExifIFD0Directory, ExifSubIfdDirectory, and several others call into ExifDirectoryBase for known tags

That is because they are all directories specified in the Exif specification. DNG tags are not.

This PR would allow a non-DNG file to start reporting DNG tags. Even if this doesn't occur in practice on the test images, I would not rule out someone hitting this issue and opening an issue about it. This library (and especially the Java implementation, which this DNG support would be migrated to) see many, many more images on a daily basis than those present in the test set.

failure to supply scope to known tag generation

This is the core of the problem, and it applies especially to TIFF files. In the case of JPEG we have clear magic bytes, and know the contained TIFF data follows the Exif specification, so we use the Exif TIFF handler code. In the case of general TIFF files, there are many different formats the file may represent. There has been quite a bit of discussion about this in other issues, mostly on the Java implementation.

Currently the TIFF handler is selected based on the file's first bytes. This appears to be inadequate in several cases. If you can find a way to identify the TIFF data as DNG at some point during the TIFF processing, we can look at doing something dynamic where we load in the appropriate TIFF tags dynamically during processing rather than at the beginning. This approach would solve quite a few problems with RAW file handling.

ongoing lack of CRLF canonicalization

I don't see this on my machine. What are your relevant Git config settings?

the .dngs in metadata-extractor-images lack DNG tags

Could you contribute a sample or two?

[twest820]

The DNG specification requires DNGs contain a DNGVersion tag. In principle, resolution of other DNG tags could be made conditional on the combination of TIFF indication and DNGVersion. If such infrastructure appears I don't mind hooking DNG up to it but I lack the context to be making design decisions around this.

I can see about a metadata-extractor-images PR with a couple samples.

[twest820]

Had a minute to look at context passing. Adding constructor overloads to ExifIfd0Directory and ExifSubIfd0Directory and calling them from the two relevant points in ExifTiffHandler might be an option.