vips-bench icon indicating copy to clipboard operation
vips-bench copied to clipboard

Convolution resampling and other libraries

Open homm opened this issue 8 years ago • 4 comments

There is a comment in pillow.py:

starting with 2.7, Pillow uses a high-quality convolution-based resize for BILINEAR ... the other systems in this benchmark are using affine + bilinear, so this is rather unfair. Use NEAREST instead, it gets closest to what everyone else is doing

I'm pretty sure that there are other libraries which do convolution resize. For example, ImageMagick and its successor GraphicsMagick. Maybe others.

Also, sips and libgd do supersampling resampling which quality is much closer to Bicubic convolution and complexity close to Bilinear convolution.

Using nearest neighbor only in Pillow unfair to other libraries. While resizing with 4-point bilinear interpolation (like in VIPS) is not suitable for scale factors < 0.5, for 0.9 (used in the test), it much closer (both quality and complexity) to Bilinear convolution rather than to nearest neighbor.

Rather than using nearest neighbor for all of this libraries, I believe it's better to use Bilinear for Pillow and add some note for VIPS, OpenCV and possibly others.

How to check

It's simple. Try to resize with scale factor 0.1. If a zoomed result will look like this, it's convolution or supersampling.

o pillow 3x

If it looks like this, it's 4-point bilinear.

o vips 3x

homm avatar Oct 05 '17 18:10 homm

libvips also has adaptive convolution resample as well as fixed window interpolators -- it has two completely different paths for the two systems.

I think that comment went in in 2015 after we swapped some mails. You said that pillow-simd always used adaptive convolution, so I wanted to avoid penalising pillow-simd, and picked nearest.

Do you think Image.BILINEAR would be fairer?

jcupitt avatar Oct 05 '17 19:10 jcupitt

libvips also has adaptive convolution resample as well as fixed window interpolators -- it has two completely different paths for the two systems.

Could you disclose why they are separated? If you have both, the good solution is to choose one of them internally based on scale factor, but do not let a user spoil the data. In my opinion, fixed window interpolation shouldn't be used for the shrinking at all, because even with scale factor from 0.5 to 1 the window width will be > 2.

Do you think Image.BILINEAR would be fairer?

Yes. To be honest, I also think that VIPS's method should be changed to adaptive convolution and OpenCV's method to INTER_AREA.

homm avatar Oct 05 '17 23:10 homm

It's just historical: vips used to just have fixed-window interpolators, and the adaptive kernel stuff was added in 8.3 (I think).

I've swapped everything to adaptive, I think.

jcupitt avatar Oct 12 '17 12:10 jcupitt

I don't think that reduce is actually adaptive:

Pillow: _out pillow

im = im.resize(0.1, kernel='linear') _out vips resize

im = im.reduce(1.0 / 0.1, 1.0 / 0.1, kernel='linear') _out vips reduce

im = im.similarity(scale = 0.1, interpolate = pyvips.Interpolate.new("bilinear")) _out vips similarity

homm avatar Oct 12 '17 15:10 homm