`load_from_memory_with_format` returns empty grayscale image after JPEG decompression (Luma16)

[KirkBuah]

I'm working on a JPEG-based codec using the image crate. The goal is to compress and decompress grayscale images with 10-bit depth. I'm storing the original image in a bit-packed format, and using Luma<u16> + DynamicImage::ImageLuma16 for JPEG encoding via encode_image.

The compression step works as expected and produces a valid JPEG. However, after decoding using load_from_memory_with_format, the resulting image is either empty or contains all zero values.

fn compress(&mut self, input: &[u8]) -> Result<Vec<u8>, String> {
    // Unpack to big-endian 16 bit buffer
    let be_buf = self.unpack_to_u16_be(input);

    // Convert u8-byte stream into a Vec<u16> so we can build an ImageBuffer<Luma<u16>, _>
    let pixels = (self.width * self.height) as usize;
    let mut samples = Vec::with_capacity(pixels);
    for chunk in be_buf.chunks_exact(2) {
        samples.push(u16::from_be_bytes([chunk[0], chunk[1]]));
    }

    // Build the image buffer and wrap it in a DynamicImage
    let gray16: ImageBuffer<Luma<u16>, Vec<u16>> =
        ImageBuffer::from_raw(self.width, self.height, samples)
            .ok_or("Failed to create ImageBuffer")?;
    let dyn_img = DynamicImage::ImageLuma16(gray16);

    // Encode jpeg
    let mut jpeg_bytes = Vec::new();
    let mut encoder = JpegEncoder::new_with_quality(&mut jpeg_bytes, self.level as u8);

    encoder
    .encode_image(&dyn_img)
    .map_err(|e| format!("Compression error: {}", e))?;
    
    Ok(jpeg_bytes)
}

fn decompress(&mut self, input: &[u8]) -> Result<Vec<u8>, String> {
    // Decode entire JPEG into a DynamicImage
    let dyn_img = image::load_from_memory_with_format(input, ImageFormat::Jpeg)
        .map_err(|e| format!("JPEG decode failed: {}", e))?;

    // Convert to 16-bit grayscale
    let gray16 = dyn_img.to_luma16();

    // Extract native-endian u16 samples and turn into big-endian bytes
    let samples: Vec<u16> = gray16.into_raw();
    let mut raw_be = Vec::with_capacity(samples.len() * 2);
    for sample in samples {
        raw_be.extend_from_slice(&sample.to_be_bytes());
    }

    // Re-pack those big-endian u16 samples back down to bd bits
    let packed = self.pack_from_u16_be(&raw_be);
    Ok(packed)
}

Note: unpack_to_u16_be and pack_from_u16_be were independently tested and are confirmed to work correctly.

Expected behavior:

After decoding a JPEG generated with encode_image from an ImageLuma16, the resulting DynamicImage (converted with .to_luma16()) should contain the original number of pixels.

Actual behavior:

gray16.into_raw() returns a Vec<u16> of all-zero values, with its length being less than the length of the original decompressed vector.

Environment:

image version: 0.25.6

Platform: x86_64-unknown-linux-gnu

Rust version: 1.87.0

[fintelia]

Our JPEG decoder is supposed to return an error when attempting to encode >8-bit per channel images: https://github.com/image-rs/image/blob/56898e0ac18fa8389b627408ffde90b85ea69899/src/codecs/jpeg/encoder.rs#L458-L475

I don't know exactly what's going wrong, but I believe it is related to DynamicImage implementing the GenericImage trait even though it really shouldn't.

[197g]

This was fixed at some point, at least to the limits of lossy 8-bit jpeg compression. (We don't provide an API with more guarantees than that right now).

Reproduction code

use image::{DynamicImage, ImageBuffer, Luma, codecs::jpeg::JpegEncoder};

struct Args {
    width: u32,
    height: u32,
    level: u8,
    samples: Vec<u16>,
}

fn main() {
    let mut args = Args {
        width: 16,
        height: 16,
        level: 10,
        samples: Vec::new(),
    };

    let data = args.encode();

    match args.decode(&data) {
        Ok(()) => {
            println!("Decoded exactly");
        }
        Err(samples) => {
            assert_eq!(args.samples, samples);
        }
    }
}

impl Args {
    pub fn encode(&mut self) -> Vec<u8> {
        // Build the image buffer and wrap it in a DynamicImage
        let samples = (0..)
            .take((self.width * self.height) as usize)
            .map(|i| i * 256 as u16)
            .collect::<Vec<u16>>();
        self.samples = samples.clone();

        let gray16: ImageBuffer<Luma<u16>, Vec<u16>> =
            ImageBuffer::from_raw(self.width, self.height, samples)
                .expect("Failed to create ImageBuffer");
        let dyn_img = DynamicImage::ImageLuma16(gray16);

        // Encode jpeg
        let mut jpeg_bytes = Vec::new();
        let mut encoder = JpegEncoder::new_with_quality(&mut jpeg_bytes, self.level);

        encoder
            .encode_image(&dyn_img)
            .map_err(|e| format!("Compression error: {}", e))
            .expect("Failed to encode image");

        jpeg_bytes
    }

    pub fn decode(&self, input: &[u8]) -> Result<(), Vec<u16>> {
        // Decode entire JPEG into a DynamicImage
        let dyn_img = image::load_from_memory_with_format(input, image::ImageFormat::Jpeg)
            .map_err(|e| format!("JPEG decode failed: {}", e))
            .expect("Failed to decode image");

        // Convert to 16-bit grayscale
        let gray16 = dyn_img.to_luma16();

        assert!(
            !gray16.iter().all(|&p| p == 0),
            "Image should not be all zeros"
        );

        let actual = gray16.into_raw();

        if self.samples == actual {
            Ok(())
        } else {
            Err(actual)
        }
    }
}

Output:

  left: [0, 256, 512, 768, 1024, 1280, 1536, 1792, 2048, 2304, 2560, 2816, 3072, 3328, 3584, 3840, 4096, 4352, 4608, 4864, 5120, 5376, 5632, 5888, 6144, 6400, 6656, 6912, 7168, 7424, 7680, 7936, 8192, 8448, 8704, 8960, 9216, 9472, 9728, 9984, 10240, 10496, 10752, 11008, 11264, 11520, 11776, 12032, 12288, 12544, 12800, 13056, 13312, 13568, 13824, 14080, 14336, 14592, 14848, 15104, 15360, 15616, 15872, 16128, 16384, 16640, 16896, 17152, 17408, 17664, 17920, 18176, 18432, 18688, 18944, 19200, 19456, 19712, 19968, 20224, 20480, 20736, 20992, 21248, 21504, 21760, 22016, 22272, 22528, 22784, 23040, 23296, 23552, 23808, 24064, 24320, 24576, 24832, 25088, 25344, 25600, 25856, 26112, 26368, 26624, 26880, 27136, 27392, 27648, 27904, 28160, 28416, 28672, 28928, 29184, 29440, 29696, 29952, 30208, 30464, 30720, 30976, 31232, 31488, 31744, 32000, 32256, 32512, 32768, 33024, 33280, 33536, 33792, 34048, 34304, 34560, 34816, 35072, 35328, 35584, 35840, 36096, 36352, 36608, 36864, 37120, 37376, 37632, 37888, 38144, 38400, 38656, 38912, 39168, 39424, 39680, 39936, 40192, 40448, 40704, 40960, 41216, 41472, 41728, 41984, 42240, 42496, 42752, 43008, 43264, 43520, 43776, 44032, 44288, 44544, 44800, 45056, 45312, 45568, 45824, 46080, 46336, 46592, 46848, 47104, 47360, 47616, 47872, 48128, 48384, 48640, 48896, 49152, 49408, 49664, 49920, 50176, 50432, 50688, 50944, 51200, 51456, 51712, 51968, 52224, 52480, 52736, 52992, 53248, 53504, 53760, 54016, 54272, 54528, 54784, 55040, 55296, 55552, 55808, 56064, 56320, 56576, 56832, 57088, 57344, 57600, 57856, 58112, 58368, 58624, 58880, 59136, 59392, 59648, 59904, 60160, 60416, 60672, 60928, 61184, 61440, 61696, 61952, 62208, 62464, 62720, 62976, 63232, 63488, 63744, 64000, 64256, 64512, 64768, 65024, 65280]
 right: [1542, 1542, 1542, 1542, 1542, 1542, 1542, 1542, 4112, 4112, 4112, 4112, 4112, 4112, 4112, 4112, 3598, 3598, 3598, 3598, 3598, 3598, 3598, 3598, 6168, 6168, 6168, 6168, 6168, 6168, 6168, 6168, 7453, 7453, 7453, 7453, 7453, 7453, 7453, 7453, 10023, 10023, 10023, 10023, 10023, 10023, 10023, 10023, 12336, 12336, 12336, 12336, 12336, 12336, 12336, 12336, 14906, 14906, 14906, 14906, 14906, 14906, 14906, 14906, 17476, 17476, 17476, 17476, 17476, 17476, 17476, 17476, 20046, 20046, 20046, 20046, 20046, 20046, 20046, 20046, 22359, 22359, 22359, 22359, 22359, 22359, 22359, 22359, 24929, 24929, 24929, 24929, 24929, 24929, 24929, 24929, 26214, 26214, 26214, 26214, 26214, 26214, 26214, 26214, 28784, 28784, 28784, 28784, 28784, 28784, 28784, 28784, 28270, 28270, 28270, 28270, 28270, 28270, 28270, 28270, 30840, 30840, 30840, 30840, 30840, 30840, 30840, 30840, 34952, 34952, 34952, 34952, 34952, 34952, 34952, 34952, 37522, 37522, 37522, 37522, 37522, 37522, 37522, 37522, 37008, 37008, 37008, 37008, 37008, 37008, 37008, 37008, 39578, 39578, 39578, 39578, 39578, 39578, 39578, 39578, 40863, 40863, 40863, 40863, 40863, 40863, 40863, 40863, 43433, 43433, 43433, 43433, 43433, 43433, 43433, 43433, 45746, 45746, 45746, 45746, 45746, 45746, 45746, 45746, 48316, 48316, 48316, 48316, 48316, 48316, 48316, 48316, 50886, 50886, 50886, 50886, 50886, 50886, 50886, 50886, 53456, 53456, 53456, 53456, 53456, 53456, 53456, 53456, 55769, 55769, 55769, 55769, 55769, 55769, 55769, 55769, 58339, 58339, 58339, 58339, 58339, 58339, 58339, 58339, 59624, 59624, 59624, 59624, 59624, 59624, 59624, 59624, 62194, 62194, 62194, 62194, 62194, 62194, 62194, 62194, 61680, 61680, 61680, 61680, 61680, 61680, 61680, 61680, 64250, 64250, 64250, 64250, 64250, 64250, 64250, 64250]