png resize artefacts related to premultiplication rounding?

[dannerei]

What are you trying to achieve?

We develop a product that relies heavily on image processing, and in particular on image resizing. We were previously using ImageMagick for this but wanted to transition to Sharp. Using Sharp 0.31.3 (or earlier), resizing (downscaling) png images with transparency works as expected (i.e the results using Sharp are almost indistinguishable from the results using ImageMagick), but using Sharp >= 0.32.0 downscaled pngs get weird color artefacts at the cross over from opaque to transparent. Our first thought was that libvips had changed something between 8.13 and 8.14, but using libvips directly proved that theory wrong. Hence, our best guess is that the change to "Prefer integer (un)premultiply for faster resizing of RGBA images", introduced in Sharp 0.32.0, is to blame for what we are seeing. If so, would it be possible to add a flag to opt out of using the integer premultiply when resizing?

When you searched for similar issues, what did you find that might be related?

https://github.com/lovell/sharp/issues/3658

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this question

sharp("path to image").resize(20, 20)

Please provide sample image(s) that help explain this question

These images are very small in size and zooming will be required to notice the artefacts I'm referring to. Larger images can be provided if needed.

Original (100x100) Resized using Sharp 0.31.3 (20x20) Resized using Sharp 0.33.3 (20x20). Notice the discoloring of the top opaque pixel row.

[lovell]

Hi, you can use pipelineColourspace to force the processing colourspace (and therefore force a bitdepth).

For example, if you want floating-point RGB, try scrgb:

sharp("path to image")
  .pipelineColourspace('scrgb')
  .resize(20, 20)
  ...

[dannerei]

That seemed to do the trick. Thanks a lot! One thing though. According to the documentation that feature is experimental, so can we rely on it?

[lovell]

that feature is experimental, so can we rely on it?

Experimental insofar as there are probably still some untested code paths, with the aim to add more test cases as bugs are found and fixed. I don't expect to remove this feature.

[dannerei]

Hi again, and an update on this:

We have now run a lot of tests using the pipelineColourspace('scrgb') setting, and although the originally reported rounding artefacts are gone, we have noticed that the resulting images look sharpened in a way that is not to our liking. Especially in transitional areas between bright and dark, the edges get unrealistically sharpened. Therefore I ask again, would it be possible to add a flag to opt out of using the integer premultiply when resizing (the 0.31.3 behaviour)?

[lovell]

we have noticed that the resulting images look sharpened in a way that is not to our liking

Please can you provide sample images and minimal code that allows someone else to reproduce. Please also include more information about expected vs actual output.

[lovell]

Aside: I've removed the "experimental" status of pipelineColourspace via commit https://github.com/lovell/sharp/commit/f67228e5ea049db67210f0c5e4b88baa6b93211c

[dannerei]

Here are a few samples using sharp 0.33.3 with and without pipelineColourspace. We expected the outputs to be very similar, but as can be seen there are major differences in the actual results.

Resize sample 1:

Original image:

Result using sharp("path to image").resize(240, 240)

Result using sharp("path to image").pipelineColourspace('scrgb').resize(240, 240)

Resize sample 2:

Original image:

Result using sharp("path to image").resize(240, 240)

Result using sharp("path to image").pipelineColourspace('scrgb').resize(240, 240)

[lovell]

Thanks for the examples, my best guess would be that the use of wide-gamut, linear scRGB is producing more accurate luminance levels in the output compared with non-linear sRGB.

If you would prefer less accurate luminance, perhaps try experimenting with gamma levels e.g. in=2.2, out=1.0.

[lovell]

@dannerei Were you able to make any progress with this?

[dannerei]

Unfortunately we were not able to reach satisfactory results in a consistent way using the proposed solution, so for now we're stuck with ImageMagick. I still hope that we can make the transition to sharp sometime in the future though, since it offers superior performance and a very pleasant API to work with.