Using affine to downscaling produces pixelated results

[ollm]

Possible bug

I am trying to resize images using affine instead of resize, in order to have access to different interpolation methods, such as nohalo, bicubic, etc. but the results look very pixelated.

Is this a possible bug in a feature of sharp, unrelated to installation?

• [x] Running npm install sharp completes without error.
• [x] Running node -e "require('sharp')" completes without error.

Are you using the latest version of sharp?

• [x] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

  System:
    OS: Linux 6.2 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (16) x64 AMD Ryzen 7 5700G with Radeon Graphics
    Memory: 22.34 GB / 31.18 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 18.17.1 - /usr/bin/node
    Yarn: 1.22.19 - /usr/bin/yarn
    npm: 9.6.7 - /usr/bin/npm
    pnpm: 8.5.1 - /usr/bin/pnpm

What are the steps to reproduce?

Using affine

(async function(){

	const sharp = require('sharp');

	const image = sharp('sample.png');
	const meta = await image.metadata();

	image
		.affine([49 / meta.width, 0, 0, 49 / meta.height], {interpolator: 'bicubic'})
		.png({compressionLevel: 9, force: true})
		.toFile('affine-bicubic-49.png')
	
})();

Image result

Using resize

(async function(){

	const sharp = require('sharp');

	const image = sharp('sample.png');
	const meta = await image.metadata();

	image
		.resize({width: 49, kernel: 'cubic'})
		.png({compressionLevel: 9, force: true})
		.toFile('resize-cubic-49.png')
	
})();

Image result

What is the expected behaviour?

Downscaling with affine should give similar results to resize

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this problem

(async function(){

	const sharp = require('sharp');

	const image = sharp('sample.png');
	const meta = await image.metadata();

	image
		.affine([49 / meta.width, 0, 0, 49 / meta.height], {interpolator: 'bicubic'})
		.png({compressionLevel: 9, force: true})
		.toFile('affine-bicubic-49.png')
	
})();

Please provide sample image(s) that help explain this problem

[lovell]

I think the current behaviour is expected. Bicubic interpolation tends to over-sharpen and can introduce ringing artefacts, which is what we're seeing here. Have you tried bilinear, which I think would be more appropriate for these images?

[ollm]

Thanks for the reply.

Yes, I have also tried bilinear and the others interpolators and they have similar results (sharp v0.33.1 and v0.26.3)

I have also tried an old version of sharp (v0.13.0), when interpolateWith was used together with resize, the results are more expected, but I don't know if this is relevant to the issue.

Sharp v0.33.1 using affine with interpolator

bicubic

bilinear

lbb

nearest

nohalo

vsqbs

Sharp v0.13.0 using resize with interpolateWith

bicubic

bilinear

lbb

nearest

nohalo

vsqbs

[lovell]

It's been a while since sharp v0.13.0 but IIRC, the interpolator provided via interpolateWith was used for the final stage of the image reduction. In the provided examples, scaling from 100 to 49 pixels would use a box shrink from 100 to 50 then affine from 50 to 49.

The modern day affine operation uses the interpolator for everything, so this means we're not really able to compare their output.

What we can do is to remove sharp from the equation and use vips at the command line.

I see the same results as sharp when e.g. running the following:

$ vips affine in.png out-bicubic.png "0.49 0 0 0.49" --interpolate=bicubic
$ vips affine in.png out-nohalo.png "0.49 0 0 0.49" --interpolate=nohalo

Are you able to experiment with vips?

[ollm]

I have tried vips directly and I get the same results as with sharp.

$ vips -v
vips-8.15.1

In the provided examples, scaling from 100 to 49 pixels would use a box shrink from 100 to 50 then affine from 50 to 49.

So, if with the current version of sharp I first apply a resize 50px and then an affine to 49px, will I get a behavior similar to interpolateWith?

Some like this:

(async function(){

	const sharp = require('sharp');

	const image = sharp('sample.png');
	const meta = await image.metadata();

	image
		.resize({width: 50, kernel: 'cubic'})
		.affine([49 / 50, 0, 0, 49 / 50], {interpolator: 'nohalo'})
		.png({compressionLevel: 9, force: true})
		.toFile('resize-affine-nohalo-49.png')
	
})();

In context, I'm implementing this in an image viewer in electron, so the input and output can be any size.