Handle HEIF images with missing file header (e.g. produced by Canon camera)

[damonlynch]

Canon CR3 images have an unusual attribute when the Canon camera is in HDR mode: according to Phil Harvey (ExifTool developer), "The embedded preview in these CR3 files are not JPG format.  They are instead basically HEIF format without the file header."

Is it is possible that libheif could be improved to be able to render these images even though they lack the file header? Alternatively, is it easy for the program calling libheif to programmatically add an appropriate file header before passing it to libheif? (I have no idea how to do that.)

The use case is to generate thumbnails from the extracted preview images of these CR3 files. (I suspect Canon's embedded HEIF previews have the same property, but I don't know how to extract them as ExifTool doesn't yet handle them).

It's likely that these type of embedded previews in Canon's CR3 files will be the "new normal" from now on. Canon tends to be pretty consistent with its file formats, in my experience.

The HEIF images are described in p. 223 of the EOS-1D X Mark II manual:

HDR PQ Settings

PQ in HDR PQ refers to the gamma curve of the input signal for displaying HDR images. HDR PQ settings enable the camera to produce HDR images conforming to the PQ specification defined in ITU-R BT.2100 and SMPTE ST.2084 (with actual display depending on monitor performance). Shots are captured as HEIF or RAW images.

• HDR stands for High Dynamic Range.
• PQ stands for Perceptual Quantization.

[lclevy]

Hi, for HDR CR3 files, HEIF metadata are inside THMB and main CRAW boxes. you can find Canon markers:

• CISZ (likely Canon Image SiZe)
• IMGD (likely IMaGe Data) and usual HEIF infos: hvcC, colr, pixi

Laurent soon documented at https://github.com/lclevy/canon_cr3

I:\dev\canon_cr3>python parse_cr3.py -v 1 "d:\cr3_samples\1dx3\struktur 1D X Mark III HEIF samples\93FG5559.CR3" filesize 0x17dbc58 00000:ftyp: major_brand=b'crx ', minor_version=1, [b'crx ', b'isom'] (0x18) 00018:moov: (0xacf0) 00020: uuid: b'85c0b687820f11e08111f4ce462b6a48' (0xa310) 00038: CNCV: b'CanonCR3_002/00.10.00/00.00.00' (0x26) ... 03758: THMB: version=1, width=320, height=214, jpeg_size=0x6bc0 (0x6bd8) 03770: b'CISZ' b'000000000000014000000140' (0x14) 03784: b'hvcC' b'0104080000009d20000000003cf000fcfefafa00000f03a0' (0xaf) 03833: colr: b'nclx' 9 10 9 80 03846: pixi: 3, 10 10 10 03856: b'IMGD' b'00006ac000006abc2601ac18c0f9558525640207952fbd1c' (0x6acc) 0a330: b'mvhd' b'00000000da72f103da72f103000000010000000100010000' (0x6c) .... 0a491: b'stsd' b'000000000000000100000170435241570000000000000001' (0x180) 00010: CRAW: (0x170) width=5472, height=3648, bits=24 0005a: b'HEVC' b'000000010000001847524944000000000000000400000101' (0x10c) 0000c: b'GRID' b'00000000000000040000010115600e40' (0x18) 00024: b'CISZ' b'0000000000000ac000000740' (0x14) 00038: b'hvcC' b'0124080000009d200000000099f000fcfefafa00000f03a0' (0xb1) 000e9: colr: b'nclx' 9 10 9 80 000fc: pixi: 3, 10 10 10 00166: b'free' b'0000' (0xa)

[t089]

Hey, I ran into the same problem. What is the correct way to make libheif understand the "H265" encoded preview image from a cr3 file?

mp4dump heif_thumb.h265
[CISZ] size=8+12
[hvcC] size=8+168
  Configuration Version = 1
  Profile Space = 0
  Profile = Rext
  Tier = 0
  Profile Compatibility = 8000000
  Constraint = 9d2000000000
  Level = 150
  Min Spatial Segmentation = 0
  Parallelism Type = 0
  Chroma Format = 2
  Chroma Depth = 10
  Luma Depth = 10
  Average Frame Rate = 0
  Constant Frame Rate = 0
  Number Of Temporal Layers = 1
  Temporal Id Nested = 1
  NALU Length Size = 4
[colr] size=8+11
[pixi] size=8+8
[IMGD] size=8+739536
[free] size=8+1

Trying to load this with libheif (via libvips) produces an error: heif: Invalid input: No 'ftyp' box: File does not start with 'ftyp' box. (2.102)

[bradh]

That really does not look like HEIF.

[t089]

Hm, thanks. I might be doing something wrong then. I am reading a file using "Libraw" and its "extract thumbnail" gives me these bytes with a type of "LIBRAW_THUMBNAIL_H265". The relevant code is here: https://github.com/LibRaw/LibRaw/blob/cc118c1c1869e2559dbd0c7639d219154cc46e40/src/postprocessing/mem_image.cpp#L133

[farindk]

h265 is not the same as HEIF. h265 is the compression format while HEIF is the file format.

The format stored in the cr3 seems to take bits and pieces from the HEIF format, but arrange it in a different way. Thus, it is not HEIF, but could maybe be transformed to HEIF by rearranging the boxes. Can you attach an example image?

[t089]

Ok makes sense, happy to share an example. This is the original CR3 file plus the "H265" frame for the preview thumbnail extracted with "libraw": https://www.dropbox.com/scl/fi/d5yt6huc8qzupu4xfhgz7/example.zip?rlkey=htcf0q4z0xtbgru9grppvkg2q&st=8ho41qfs&dl=0

[kmilos]

I guess the fundamental problem is that the raw CR3 file is itself an ISOBMFF one, so it didn't make sense (or might not even possible) to embed another complete one inside?