strukturag
Files with shared auxiliary images fail to parse
Filing issue to document email thread with @farindk.
ImageMagick is rejecting some prototype files that I have been creating with the following error: identify: Invalid input: Unspecified: Too many auxiliary image references (2.0) `file_with_shared_alpha.heic' @ error/heic.c/IsHEIFSuccess/142.
The error is coming from libheif rather than ImageMagick itself. The file in question has two main images (in an 'altr' group) that share an alpha auxiliary image. To me this should be perfectly valid and makes a lot of sense once you start creating files containing 'altr' groups, but I can see why this would be unexpected for simpler files.
The conclusion from the thread is that there is nothing in the spec that explicitly forbids sharing the same auxiliary image between multiple main images and that the restriction in libheif should probably be relaxed. ISOBMFF says that all irefs of a specific type from a source item should be collected into a single ItemReferenceBox, so having multiple irefs is not really an option. file_with_shared_alpha.heic.zip
It looks like we just need to add the auxiliary image to each of the parents. Might be some complexity with loop detection, but will check that.
One question before I look to PR this: would it make more sense to have the altr group (in this test file, item 4) as the target? Is that at least a possibility that should be covered / supported?
@bradh
would it make more sense to have the altr group (in this test file, item 4) as the target? Is that at least a possibility that should be covered / supported?
Good question. It would only be possible under the unif brand, at which point it falls under the fuzziness of if the semantics of the reference permit it.
While I think the intent of the writer is relatively clear if you let the auxl point to the altr group, I think it's much clearer to let the auxiliary image reference the main image items directly. This avoids all the ambiguity that you get with the unif brand (and works even for parsers that don't implement groups).
I think it's fine to support reading files that have an auxl reference to an altr group, but we should probably discourage such files from being created.
The same issue might also be present for shared metadata and shared thumbnails. Creating a file with shared EXIF payloads seems to result in the following error from ImageMagick:
identify: Invalid input: Unspecified: Metadata not correctly assigned to image (2.0)
(Assuming I didn't mess up creating the file)
Attached a file that shares alpha, thumbnail and XMP between the two main images in an altr group.
file_with_shared_everything.heic.zip
I've got it to a "less broken" stage:
$ ./build/examples/heif-info ./file_with_shared_everything.heic
MIME type: image/heic
main brand: heic
compatible brands: heic, mif1
image: 512x384 (id=1), primary
colorspace: YCbCr, 4:2:0
bit depth: 8
thumbnail: 128x96
color profile: no
alpha channel: yes
depth channel: no
metadata:
XMP: 356 bytes
transformations:
none
region annotations:
none
properties:
image: 1024x768 (id=3)
colorspace: YCbCr, 4:2:0
bit depth: 8
thumbnail: 128x96
color profile: no
alpha channel: no
depth channel: no
metadata:
XMP: 356 bytes
transformations:
none
region annotations:
none
properties:
and ImageMagick is more on-board with this:
$ identify -verbose ./file_with_shared_everything.heic
Image:
Filename: ./file_with_shared_everything.heic
Format: HEIC (Apple High efficiency Image Format)
Mime type: image/x-heic
Class: DirectClass
Geometry: 512x384+0+0
Units: Undefined
Colorspace: YCbCr
Type: TrueColor
Base type: Undefined
Endianness: Undefined
Depth: 8-bit
Channel depth:
red: 8-bit
green: 8-bit
blue: 8-bit
Channel statistics:
Pixels: 196608
Red:
min: 0 (0)
max: 255 (1)
mean: 107.035 (0.419743)
standard deviation: 55.2343 (0.216605)
kurtosis: -0.385301
skewness: 0.357236
entropy: 0.969307
Green:
min: 33 (0.129412)
max: 154 (0.603922)
mean: 115.458 (0.452778)
standard deviation: 12.9742 (0.0508791)
kurtosis: 1.90072
skewness: -1.07821
entropy: 0.822435
Blue:
min: 103 (0.403922)
max: 169 (0.662745)
mean: 125.525 (0.492253)
standard deviation: 5.94608 (0.023318)
kurtosis: 2.07753
skewness: 0.88771
entropy: 0.748516
Image statistics:
Overall:
min: 0 (0)
max: 255 (1)
mean: 116.006 (0.454925)
standard deviation: 24.7182 (0.0969341)
kurtosis: 3.09398
skewness: -0.200445
entropy: 0.846753
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33)
green primary: (0.3,0.6)
blue primary: (0.15,0.06)
white point: (0.3127,0.329)
Background color: ycbcr(255,255,255)
Border color: ycbcr(223,223,223)
Matte color: ycbcr(189,189,189)
Transparent color: ycbcr(0,0,0)
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 512x384+0+0
Dispose: Undefined
Iterations: 0
Compression: Undefined
Orientation: TopLeft
Properties:
date:create: 2024-03-23T09:25:46+00:00
date:modify: 2024-03-20T11:58:28+00:00
signature: 92c65b443b025008d972d4a002b234cc750bcc5535bfc2f73c99def327e5c49a
Artifacts:
filename: ./file_with_shared_everything.heic
verbose: true
Tainted: False
Filesize: 0B
Number pixels: 196608
Pixels per second: 5.75156MB
User time: 0.040u
Elapsed time: 0:01.034
Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org
The alpha still isn't properly assigned to the second image. Need to go again on the scaling.
Have updated the implementation.
./examples/heif-info ../file_with_shared_everything.heic
MIME type: image/heic
main brand: heic
compatible brands: heic, mif1
image: 512x384 (id=1), primary
colorspace: YCbCr, 4:2:0
bit depth: 8
thumbnail: 128x96
color profile: no
alpha channel: yes
depth channel: no
metadata:
XMP: 356 bytes
transformations:
none
region annotations:
none
properties:
image: 1024x768 (id=3)
colorspace: YCbCr, 4:2:0
bit depth: 8
thumbnail: 128x96
color profile: no
alpha channel: yes
depth channel: no
metadata:
XMP: 356 bytes
transformations:
none
region annotations:
none
properties:
