Add support for sharp yuv

[maryla-uc]

As a follow up to #586 (smooth chroma upsampling), a second (bigger) step would be to add support for "sharp yuv", a slower rgb->yuv420 conversion technique that minimizes arifacts caused by chroma subsampling. The algorithm assumes that the image is decoded with bilinear upsampling.

It was first implemented in webp, see an intro/example here https://www.ctrl.blog/entry/webp-sharp-yuv.html libwebp code is here: https://github.com/webmproject/libwebp/blob/36a6eea3bc2871c7f0166193a6fa42241af4a464/src/enc/picture_csp_enc.c#L447

Would you consider adding it as an optional subsampling method to libheif? If so, would you rather have it be reimplemented in libheif, or use an implementation from an external library, if available? (we are considering adding it to libyuv or possibly creating a separate library).

[farindk]

Do you know which algorithm is used by SharpYUV ? I don't think it is simple bilinear upsampling. Maybe bicubic or Lanczos, or something else?

[maryla-uc]

SharpYuv is a subsampling technique but it assumes the upsampling is bilinear (you can see it here: https://github.com/webmproject/libwebp/blob/36a6eea3bc2871c7f0166193a6fa42241af4a464/src/dsp/yuv.c#L229 )

SharpYuv is an iterative process. Basically, it starts with the naive subsampled version (first candidate), upsamples it with bilinear the way it expects the decoder to do, and subsamples it again. It checks how far the result is from the target (= subsampled original), shifts the candidate by that much, and does another iteration.

This is a bit of a simplification but it's the gist of it. The fact that it's iterative means it's slower than regular subsampling. This is why it's not enabled by default in cwebp, although with more complex image formats/slower encoders, the overhead might be less noticeable.

[farindk]

Understood, thanks for the explanation.

Sounds like this could be done with a large matrix inversion. Since it's an overdetermined system, one might look for a least squares solution. Maybe the webp code is just doing that iterative inverse. I didn't have a close look yet.

[tongyuantongyu]

I did some work on porting sharp yuv to libavif (https://github.com/AOMediaCodec/libavif/pull/444), and I'd like to share some information here.

this could be done with a large matrix inversion

Unfortunately this is not the case here. @maryla-uc's explanation mentioned that "checks how far the result is from the target", and this step involves srgb transfer characteristic function (in webp's case. For heif/avif, other TCs in H.273 may also be used) and introduces non-linearity, hence why iterative method is used.