libheif
libheif copied to clipboard
strukturag
Regression since 925c04a: lossless is no longer lossless, --matrix_coefficients=0 is using YCbCr, no nclx
Indeed, https://github.com/pphh77/libheif-Windowsbinary/releases/download/v1.9.1/libheif-1.9.1-win64.7z produces this sample https://github.com/strukturag/libheif/files/6850705/output1.zip that is lossless compared to (I am using -p lossless=true --matrix_coefficients=0 -p chroma=444) https://user-images.githubusercontent.com/515193/126373113-4d937cc4-4963-407f-bb54-19b8560a36b4.png
Now with 1.12.0 it is like this: https://github.com/strukturag/libheif/issues/62#issuecomment-883592202
I will also point out THAT ONE MUST specify --matrix_coefficients=0 --transfer_characteristics 13 --color_primaries 1 for it to be sRGB, but that does not work. Without that there is no way to say whether it is XYZ or RGB! Or even what RGB that is, RGB DCI-P3, DCI-D65, other primaries RGB, et cetera. One can even by MISTAKE set 2.2 transfer charact. for sRGB, which IS WRONG!
Bisecting shows 925c04a37f194ca992f5e5a9ee98ffc1b597dbaa as the first bad commit.
Is the image immediatelly on the regressed commit the same as in HEAD of master (bitstream and/or decoded png)? Because if yes, then I suppose it may be just a problem of noop if both are sRGB or a problem of what is written in nclx (as I said it MUST be --matrix_coefficients=0 --transfer_characteristics 13 --color_primaries 1).
Is the image immediatelly on the regressed commit the same as in HEAD of master (bitstream and/or decoded png)?
Yes.
Some intermediate commits during bisection showed other glitches, but the first regressed commit produces the exact same round-tripped image as master and 1.12.0 do.
FWIW nclx is not getting written for HEIC at all: running heif-info shows “color profile: no”, and I don’t see nclx in heif-info -d. In contrast, AVIF produces “color profile: nclx”; but the net result is still the same broken round trip.
--transfer_characteristics 13 --color_primaries 1
I’ve just checked, just in case. Adding these flags made no difference.
I expected this: neither of these properties affects simple RGB round-trip; none of the software involved does colour correction for RGB primaries or gamma compression, so these properties never come into play. FWIW I think heif-enc entirely ignores the source PNG’s gAMA/sRGB chunks and possibly copies any ICC profile verbatim, and heif-convert does not write gAMA/sRGB to an output PNG and possibly copies any ICC profile verbatim. Similarly, ImageMagick’s compare and convert-to-PPM do not, by default, do any colour correction.
I believe --transfer_characteristic and --colour_primaries are merely saved as given: heif-enc assumes the user is telling it what the source is, not asking it to convert.
Without that there is no way to say whether it is XYZ or RGB! Or even what [kind of] RGB that is
Note: this is no different from any other image format; say, PNG. Without an embedded colour profile (or at least a sRGB chunk or gAMA+cHRM), it is impossible to know exactly what colours are represented. Most pure-RGB processing chains don’t need to know this and simply don’t do colour correction. When colour correction is desired for display, usually, either sRGB or the output device’s native physical properties are assumed.
Just tried encoding to AVIF with libheif and decoding with libavif to exclude libheif’s decoder from the equation: makes no difference; the round trip is still lossy. Curiously, avifdec reports:
* Matrix Coeffs. : 6
which leads me to suspect that heif-enc actually ignores --matrix_coefficients=0 entirely (and lossily converts the input to BT.601 YCbCr).
which leads me to suspect that
heif-encactually ignores--matrix_coefficients=0
Yeah. Microsoft's thumbnailer actually shows pinky image for RGB (correctly lossless) image I attached. But it does not for 1.12.0 (same happens for some other stuff). Alas, without patches to ffmpeg supporting HEIC it is hard to tell whether it is YCbCr or RGB.
actually ignores
--matrix_coefficients=0entirely
Yes. It is bitperfect with --matrix_coefficients=6 in 1.12.0. Just wonderful. Also it no longer writes nclx atom (my file has one).
Currently no free time for this...
You can just revert those two above mentioned commits.
I can reproduce this issue as well with libheif 1.12.0
$ heif-enc -p lossless=true -p chroma=444 --matrix_coefficients=0 DSC_0209.png DSC_0209.heic
$ djxl DSC_0209.heic DSC_0209.heic.png
$ compare -quiet -metric SSIM DSC_0209.png DSC_0209.heic.png
0.996387
Manually build 1.11.0, and the result is lossless but with bigger size of file as we expected.
Here is my experiment results comparing PNG, HEIC, JPEG XL.
- Uncompress BMP: 100MB
- PNG: 32.3MB
- HEIC lossless mode with 1.12.0: 20.1MB, SSIM = 0.996387
- HEIC lossless mode with 1.11.0: 32.9MB, SSIM = 1.0
- JPEG XL lossless mode: 21.5MB, SSIM = 1.0
JPEG XL achieves very amazing result!
- PNG: 32.3MB
Recheck with ffmpeg -i file.bmp -compression_level 9 -pred mixed out.png
And bmp can have good compression too.
- PNG: 32.3MB
Recheck with ffmpeg -i file.bmp -compression_level 9 -pred mixed out.png
And bmp can have good compression too.
I used imagemagick for png conversion with compression level 9. I believe ffmpeg is using the same libpng library.
Main advantage comes from mixed algorithm. And no, ffmpeg does not use libpng, just like Photoshop. In the past libpng was part of the tree but it was modifed many times since then. Please check it and check this, it describes some other PNG compressors: https://siipo.la/blog/whats-the-best-lossless-image-format-comparing-png-webp-avif-and-jpeg-xl
(In reply to @dbtsai from this comment)
$ djxl DSC_0209.heic DSC_0209.heic.png
djxl (from libjxl) only decodes JXL files. How did you manage to use libjxl to decode DSC_0209.heic to DSC_0209.heic.png?
Is the djxl in the command a typo? Or are you using a fork of libjxl which supports decoding HEIC?
Hi @farindk, may I draw your attention to this bug which was caused by https://github.com/strukturag/libheif/commit/925c04a37f194ca992f5e5a9ee98ffc1b597dbaa.
I agree with @ValZapod that this commit is problematic and should be fixed or reverted (but it's probably too late to revert it given that it introduced changes to the public API). The commit comments out the call to heif_image_set_nclx_color_profile(), and instead introduces an option called output_nclx_profile which is completely unused. The extra code in heif_context_encode_image is also a noop.
As a result, the --matrix_coefficients, --colour_primaries and --transfer_characteristic flags of heif-enc have zero effect ever since this commit, which in turn prevents achieving true lossless since RGB is always converted to YUV.
In https://github.com/strukturag/libheif/issues/407#issuecomment-756353553 you said "the color conversion used the same matrix_coefficients for the YCbCr->RGB conversion as for the final RGB->YCbCr while it should only be used for output". I'm not sure what the reasoning is behind this. To me, it seems that most of the time you want the RGB->YUV transformation to be the same (but inversed, obviously) as the YUV->RGB one in order to retrieve the original pixels. Therefore the original behavior seemed fine to me.
The simplest fix is to just uncomment the call to heif_image_set_nclx_color_profile() in heif_enc.cc which brings back the old behavior.
As a result, the --matrix_coefficients, --colour_primaries and --transfer_characteristic flags of heif-enc have zero effect ever since this commit, which in turn prevents achieving true lossless since RGB is always converted to YUV.
that explains why nothing changes when I try to set different values there...
