Ultra HDR support

[cjee21]

Newer Android phones capture images in Ultra HDR. https://developer.android.com/media/platform/hdr-image-format

Samples: https://www.dpreview.com/sample-galleries/7614427312

[cjee21]

Apple also has it: https://www.androidauthority.com/google-apple-hdr-photo-standard-3495035/

[JeromeMartinez]

I have a HEIC file from iPhone 15 with the gain map stuff (and a grid system, I need to implement that too, I started the grid support), now it needs an implementation... It seems that the HEIC and the JPG implementation are very different, yet other work to do...

[cjee21]

More samples (ISO 21496-1?) here: https://helpx.adobe.com/camera-raw/using/gain-map.html

[JeromeMartinez]

More samples (ISO 21496-1?)

"currently under development."

but there is a start of spec: Gain Map Specification (1.0 draft 15, February 2024)

As a quick check, it is an additional picture + XML metadata, at least in HEIC, I need to find where is the additional picture in the JPEG file.

[cjee21]

Looks like there are multiple formats, the Android Ultra HDR XMP, Adobe XMP, Apple XMP and the under-development ISO 21496-1 that they all are going to support too.

[cjee21]

JPEG uses MPF to store the additional image. I wonder if Sony's old 3D photos also uses this.

[cjee21]

If want to handle MPF, it's IFDs like Exif and TIFF again...

[cjee21]

Sony's 3D photos also uses MPF containing 2 to 15 images.

https://web.archive.org/web/20120123035845/http://www.stereomaker.net/sony/sony.htm

[cjee21]

Found the MPF

00EC51 FFE2 - APP2 - Application-specific marker 2 - Multi-Picture Format (90 bytes)
00EC51  Header (4 bytes)
00EC51   Marker:                               65506 (0xFFE2)
00EC53   Fl - Frame header length:             88 (0x0058)
00EC55  Signature:                             MPF
00EC59  Header (8 bytes)
00EC59   Alignment:                            II*
00EC5D   Offset to first IFD:                  8 (0x00000008)
00EC61  MP Index IFD (2 bytes)
00EC61   Count:                                3 (0x0003)
00EC63  (Not parsed):                          (72 bytes)

[JeromeMartinez]

Found the MPF

Do you plan to do a PR and/or need help?

[cjee21]

Do you plan to do a PR and/or need help?

What's the plan? Another File_MPF_cpp to handle the IFD parsing like Exif?

[JeromeMartinez]

Another File_MPF_cpp to handle the IFD parsing like Exif?

I guess so. With a public Exif class member for storing offset & size so the JPEG parser can seek to the next image when a SOS is reached.

[cjee21]

Android 16 adds HDR PNGs

https://source.android.com/docs/core/graphics/hdr-screenshots#android16-hdr-shot

I cannot find a sample yet.

[cjee21]

Now that we already parse all the XML and MPF used for Ultra HDR, is there a need to detect and display the Ultra HDR format?

[JeromeMartinez]

Now that we already parse all the XML and MPF used for Ultra HDR, is there a need to detect and display the Ultra HDR format?

Definitely :-p.

[cjee21]

This image contains lots of unparsed data at the end. Looks like that is where the original image and/or depth map and/or confidence map is. The gain map is in MPF. Full parse mode shows additional images containing Exif and XMP data at the end. Text output is messed up in full parse mode.

https://www.dpreview.com/sample-galleries/7614427312/google-pixel-9-pro-and-pro-xl-sample-gallery/9931408803

[cjee21]

This one portrait mode has more: https://www.dpreview.com/sample-galleries/7614427312/google-pixel-9-pro-and-pro-xl-sample-gallery/0809396176

Looks like it even has more than one unrelated MPFs.

[cjee21]

Some HDR JPEG samples here. Probably using the Adobe style.

https://gregbenzphotography.com/hdr-gain-map-gallery/

[cjee21]

Android 16 adds HDR PNGs

https://source.android.com/docs/core/graphics/hdr-screenshots#android16-hdr-shot

I cannot find a sample yet.

PNG with gainmap can be found at https://github.com/google/skia/tree/main/resources/images

[cjee21]

Sample with both Adobe / Ultra HDR and ISO 21496-1 metadata https://cdn.mos.cms.futurecdn.net/Wc2Jv4MwN5Xu5G4oH8zPX9.jpg

Sample with Adobe / Ultra HDR metadata + lots of camera-specific metadata in XMP https://fdn.gsmarena.com/imgroot/reviews/25/sony-xperia-1-vii/rev13/camera/gsmarena_1101.jpg

[cjee21]

Sample with both Apple and ISO 21496-1 metadata https://fdn.gsmarena.com/imgroot/reviews/24/apple-iphone-16-pro-max/camera/gsmarena_1102.jpg

[cjee21]

Sample with both Apple and ISO 21496-1 metadata https://fdn.gsmarena.com/imgroot/reviews/24/apple-iphone-16-pro-max/camera/gsmarena_1102.jpg

For Apple type of HDR, it appears there is data in Exif Apple MakerNote (33/HDR Headroom, 48/HDR Gain) of primary image as well as in XMP (HDRGainMapVersion, HDRGainMapHeadroom in HDRGainMap namespace) of 2nd MPF image.

HDRGainMapHeadroom in the XMP can be derived from the MakerNote values.

https://developer.apple.com/documentation/appkit/applying-apple-hdr-effect-to-your-photos

[cjee21]

Pixel 10 Pro XL sample with Ultra HDR XMP and C2PA but looks like no ISO 21496-1... https://fdn.gsmarena.com/imgroot/reviews/25/google-pixel-10-pro-xl/camera/gsmarena_004.jpg