fast_image_resize icon indicating copy to clipboard operation
fast_image_resize copied to clipboard

Downscaling in linear color space?

Open virtualritz opened this issue 2 years ago • 4 comments

I can't find anything in the documentation (or the source) that indicates that there is any support for conversion of data into a linear color space during resizing.

This is required to get correct results when downscaling images that are not in a linear color space. I.e. any 24/32bit PNG, 24bit JPG etc. one would find in the wild.

While this linearization could be done outside the resizing code (and thus this crate) for the supported i32 and f32 types, for the u8 case this is not possible. If the crate user linearizes the input data from some color space u8 to linear u8 there is too much loss of information – banding/posterization ensues. See example images in above linked article. For this reason the linearization before the filtering and de-linearization after has to be part of the resizing process itself for u8 grayscale or [u8; 3] RGB.

Would it be possible to add support for this? Basically u8 data needs to be linearized to at least u16, better u32 or f32, resized, de-linearized and stored back into u8. And the color space can't be baked into the crate. I.e. you can not just assume sRGB or gamma 2.2 if you want to do this properly.

Possibly through feature-flag gated support for e.g.:

  • user-specified closures for transforming into/from linear color space during filtering
  • a crate like colstodian

In case of closures the closure should have access to all components of a color so it can convert to/from other spaces, e.g. RGB to Lab, if the user wishes the resize to happen in that space.

virtualritz avatar Dec 22 '21 16:12 virtualritz

The current implementation does nothing special with pixel components and knows nothing about color space of image.

The main feature of this crate is the use of SIMD instructions of CPU to speed up image resizing. So it is most likely not possible to simply use closures for pixel transforming. Because this will significantly slow down the resizing.

I'll can try to add support of images with u16 components of pixel (U16, U16x3 and U16x4). But I don't promise that I will do it soon.

Cykooz avatar Dec 22 '21 18:12 Cykooz

It doesn't need to be closures. It can be something based on macros so users can have compile-time optimized paths for common encodings like sRGB or Adobe RGB.

Without this the the crate is useless for downscaling u8 component images that are non-linear (100% of u8 component images that are photos, textures etc., for obvious reasons) since the results will be wrong.

I would put a note in the README at least.

virtualritz avatar Dec 28 '21 12:12 virtualritz

I've added support of U16x3 pixels. For now without optimizations using SIMD.

Cykooz avatar Jan 27 '22 17:01 Cykooz

v0.8.0

  • Added optimisation for convolution of U16x3 images with helps of SSE4.1 and AVX2 instructions.

Cykooz avatar Mar 23 '22 21:03 Cykooz

In version 2.0.0 I added PixelComponentMapper structure that allows you to create colorspace converters for images whose pixels based on u8 and u16 components.

Source and destination images for mapper may have different bit depth of one pixel component. But count of components must be equal. For example, you may convert U8x3 image with sRGB colorspace into U16x3 image with linear colorspace.

In addition, the crate contains functions create_gamma_22_mapper() and create_srgb_mapper() to create instance of PixelComponentMapper that converts images from sRGB or gamma 2.2 into linear colorspace and back.

You may use PixelComponentMapper to convert image colorspace before and after resizing.

Cykooz avatar Oct 28 '22 18:10 Cykooz