eye-rs icon indicating copy to clipboard operation
eye-rs copied to clipboard

Add `image` crate support

Open fenjalien opened this issue 3 years ago • 3 comments

I feel implementing some integration with the image crate would be a good move. I've just been trying out eye and wanted to save a captured frame to an image file which leads to something like this:

image::ImageBuffer::<image::Rgb<u8>, &[u8]>::from_raw(stream_desc.width, stream_desc.height, frame.as_bytes()).unwrap().save("image.png")?;

It could be as simple as having a to_image_buffer method on the Frame struct. Or more extreme replacing Frame entirely and only returning ImageBuffer(or something similar I'm not well versed with it yet). You could even use image's pixel traits instead of PixelFormat?

fenjalien avatar May 01 '21 10:05 fenjalien

I thought about this before and generally feel the same way. The problem is that the image crate is too restrictive and cannot handle arbitrary pixel formats all that well. For example, YUV formats such as YUYV which is very common among consumer grade hardware is entirely unsupported. The pixel traits of the image crate are thus insufficient.

raymanfx avatar May 03 '21 11:05 raymanfx

@raymanfx maybe you can help with the conversion of the frame to the image where I can save it. I'm having some problems with converting. I'm trying to specify specs

use eye_hal::{ stream::Descriptor as StreamDescriptor }
let stream_params = StreamDescriptor {
        width: 320,
        height: 240,
        pixfmt: PixelFormat::Rgb(24),
        interval: std::time::Duration::from_nanos(33333335),
    };

Then taking the frame

let mut stream = camera.start_stream(&stream_params)?;
   let frame = stream
       .next()
       .expect("Stream is dead")
       .expect("Failed to capture frame"); 

use image::{ImageBuffer, Rgb};
let res =
       ImageBuffer::<Rgb<u8>, &[u8]>::from_raw(stream_params.width, stream_params.height, &frame)
           .unwrap()
           .save(file_path);

But when I do save the image it has multiple horizontal lines, and it seems that the PixelFormat is not applied.

hadhoryth avatar Apr 18 '23 15:04 hadhoryth

@hadhoryth I assume you are on Linux, thus using the v4l platform backend, is that correct?

The backend invokes set_format() here: https://github.com/raymanfx/eye-rs/blob/master/eye-hal/src/platform/v4l2/device.rs#L196. This boils down to the implementation in v4l-rs here: https://github.com/raymanfx/libv4l-rs/blob/master/src/video/macros.rs#L150.

As you can see, the implementation negotiates the pixelformat with the Linux kernel (because that's what v4l2 is designed to do). The system call will return the actual format in use. The problem is that we don't propagate that back up the callchain right now. v4l-rs already does this - we just have to make use of the returned value in eye-rs.

Clearly, this is an API issue and needs to be fixed. Do you want to create a separate issue for tracking that item? I will try to have a look at it this week.

raymanfx avatar Apr 27 '23 07:04 raymanfx