Image format converting questions

[LightToYang]

I am working on speeding up image loading for training models on ImageNet Dataset.

Based on https://stackoverflow.com/questions/57663734/how-to-speed-up-image-loading-in-pillow-python,

#!/usr/bin/env python3 

import numpy as np 
import pyvips 
import cv2 
from PIL import Image 

def usingPIL(f): 
    im = Image.open(f) 
    return np.asarray(im) 

def usingOpenCV(f): 
    arr = cv2.imread(f,cv2.IMREAD_UNCHANGED) 
    return arr 

def usingVIPS(f): 
    image = pyvips.Image.new_from_file(f) 
    mem_img = image.write_to_memory() 
    imgnp=np.frombuffer(mem_img, dtype=np.uint8).reshape(image.height, image.width, 3)  
    return imgnp 

def usingPILandShrink(f): 
    im = Image.open(f)  
    im.draft('RGB',(1008,756))  
    return np.asarray(im) 

%timeit usingPIL('image.jpg')
315 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit usingOpenCV('image.jpg')
102 ms ± 1.5 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit usingVIPS('image.jpg')
8.44 ms ± 31.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

%timeit usingPILandShrink('image.jpg')
77.2 ms ± 994 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

I find pyvips is suitable for my tasks. Since, the image formats of ImageNet contain RGB, Gray and CYMK, how can I transform Gray or CYMK format images to RGB format like https://libvips.github.io/pyvips/intro.html#numpy-and-pil

import pyvips
import numpy as np

format_to_dtype = {
    'uchar': np.uint8,
    'char': np.int8,
    'ushort': np.uint16,
    'short': np.int16,
    'uint': np.uint32,
    'int': np.int32,
    'float': np.float32,
    'double': np.float64,
    'complex': np.complex64,
    'dpcomplex': np.complex128,
}

img = pyvips.Image.new_from_file(sys.argv[1], access='sequential')
np_3d = np.ndarray(buffer=img.write_to_memory(),
                   dtype=format_to_dtype[img.format],
                   shape=[img.height, img.width, img.bands])

[jcupitt]

Hello @LightToYang,

I would use something like:

    image = pyvips.Image.new_from_file(f, access="sequential")
    image = image.colourspace("srgb") 
    mem_img = image.write_to_memory() 
    imgnp=np.frombuffer(mem_img, dtype=np.uint8).reshape(image.height, image.width, 3)  
    return imgnp 

And you should get an RGB buffer.

[jcupitt]

However, I think your benchmark is misleading. libvips will cache the JPEG decode, so it's just being decoded once. Try:

def usingVIPS(f): 
    image = pyvips.Image.new_from_file(f, access="sequential") 
    mem_img = image.write_to_memory() 
    imgnp=np.frombuffer(mem_img, dtype=np.uint8).reshape(image.height, image.width, 3)  
    return imgnp 

This test is just timing JPEG decode, so all the libraries here will be almost the same speed, since they are using the same JPEG decode library.

pyvips can be much quicker for some image files -- regions from tiled tiff, for example, or large whole-slide images -- but I don't know if that will apply here.

[jcupitt]

Actually, it's still a bit quicker. With this code:

#!/usr/bin/python3 

import numpy as np 
import pyvips 
import cv2
from PIL import Image 
import timeit

def usingPIL(f): 
    im = Image.open(f)
    return np.asarray(im) 

def usingOpenCV(f): 
    arr = cv2.imread(f,cv2.IMREAD_UNCHANGED)
    return arr 

def usingVIPS(f):
    image = pyvips.Image.new_from_file(f, access="sequential")
    image = image.colourspace("srgb")
    mem_img = image.write_to_memory() 
    imgnp=np.frombuffer(mem_img, dtype=np.uint8).reshape(image.height, image.width, 3)  
    return imgnp 

def usingPILandShrink(f): 
    im = Image.open(f)  
    im.draft('RGB',(1512,1008))
    return np.asarray(im) 

def usingVIPSandShrink(f):
    image = pyvips.Image.new_from_file(f, access="sequential", shrink=4)
    image = image.colourspace("srgb")
    mem_img = image.write_to_memory()
    imgnp=np.frombuffer(mem_img, dtype=np.uint8).reshape(image.height, image.width, 3)
    return imgnp

def bench(name):
    result = timeit.timeit(f"using{name}('image.jpg')", 
                           setup=f"from __main__ import using{name}",
                           number=10)
    print(f"using{name}: {result * 10} ms")

bench("PIL")
bench("OpenCV")
bench("VIPS")
bench("PILandShrink")
bench("VIPSandShrink")

Running on ubuntu19.10 I see:

$ ./bench2.py 
usingPIL: 24.930770930077415 ms
usingOpenCV: 31.592376419866923 ms
usingVIPS: 14.631749049876817 ms
usingPILandShrink: 4.895634700078517 ms
usingVIPSandShrink: 6.390921729907859 ms

[jcupitt]

Update

pyvips now has better numpy integration, so you can just write:

def usingVIPSandShrink(f):
    image = pyvips.Image.new_from_file(f, access="sequential", shrink=4)
    return image.numpy()