Optionally return timestamp and frame sequence from the Capture method

[slovak194]

Hi!

I'm working with a multicamera setup and my application needs a certain level of synchronization between frames from individual cameras. My cameras have a hardware synchronization feature but as captures are done in separate threads I need access to the sequence and timestamp of each frame to group them into synchronized chunks.

It is possible to take timestamp and sequence fields from v4l2_buffer if I use v4l2Camera class from this repository, but it looks like it is going to be deprecated in favor of gstCamera which provides a convenient videoSource API.

Are there any plans or workarounds to provide the aforementioned functionality for gstCamera? That sounds like a good idea, but I cannot find any similar issue. Am I the only one who needs a timestamp to be returned together with the image for further alignment/synchronization? :)

Best regards, Alex

[ricknitsche]

I agree, this would be very useful!

[Stack-it-up]

Do you have any news about this? I'm working on something similar (RealSense camera with depth data to be processed on CPU)

[dusty-nv]

@Stack-it-up after you do videoSource::Capture(), you can use videoSource::GetLastTimestamp() and it will give you the timestamp of the image that it just returned. These timestamps are pulled from the incoming GStreamer buffers here. When using a v4l2src, I'm not sure where GStreamer gets them from, whether it be the v4l2_buffer or somewhere else internally.

If you are using Python with cudaImage, you can use img.timestamp and it will give you the same timestamp.

[Stack-it-up]

thank you @dusty-nv : I think I will try this as a fallback since I noticed that directly pulling the frame from the rs2::pipeline and then "copying" the data to some cudaMallocManaged buffer seems to be faster than using the actual Gstreamer pipeline.