Complete Workflow for Converting RGB Video to Event Data

[JeremyCJM]

Hi supitalp,

Thanks a lot for your work! I finally installed your package successfully and got the 'out.bag' output. But I am not sure what is this output and how to use and visualize it.

I am wondering if it is possible to give us a complete explanation of the workflow for how to convert a RGB video to the event data using your simulator.

Looking forward to hearing from you.

Thanks, Jeremy

[supitalp]

Hi Jeremy, Thanks for your feedback. Since this question has been asked several times already, I just wrote a quick tutorial of the workflow here: Simulating events from a video.

Before trying this guide, please pull and recompile esim as I updated a few minor things to make it easier to run this tutorial. Please let me know if that helps, or if you have any specific question and I'll update this guide.

Henri

[JeremyCJM]

Thanks a lot for the tutorial! It helps.

Jeremy

[JeremyCJM]

Hi Henri,

After I went through the workflow in your tutorial, I successfully got the 'out.bag', but I could not get the event visualization. Specifically,

1. rosrun dvs_renderer dvs_renderer events:=/cam0/events After running this, it got stuck. Is it normal? Does 'events:=/cam0/events' mean that the event data will be saved in /cam0/events?
2. rosbag play /tmp/out.bag I ran it in another terminal. This command is successfully executed.
3. rqt_image_view /dvs_rendering After running this, it opened a window, but there are no event visualizations showed in tutorial.

After finishing all the command, I checked there is no folder /cam0/events. Do you know how to fix it?

Another question is how to get the raw (x, y, time, polarity) event data?

Looking forward to your reply.

Thanks, Jeremy

[supitalp]

Hi Jeremy,

1. ESIM uses the framework ROS to stream different sensor measurements (images, events, etc.) over different "lines", which are called "topics". Each "topic" has a name (e.g. /cam0/events). I suggest that you first go through the ROS beginner tutorials to get a basic understanding of ROS, that should greatly help to use ESIM.

2. It might be that you opened the window after the rosbag has finished playing, in which case you will not see any output. You can loop rosbag playing using the -l option:

rosbag play /tmp/out.bag

and also optionally slow it down (e.g. by a factor of 10) with the option -r:

rosbag play /tmp/out.bag -l -r 0.1

4. To access the raw event data in (x, y, time, polarity) form, you can implement your own ROS node that will listen to the /cam0/event topic (an example such node is the dvs_renderer node that just displays events on an image). Another possibility would be to extract the events from the rosbag and save them as a text file. I'll post an example script on how to do that shortly.

[JeremyCJM]

Hi Henri,

I succeeded this time. Thanks for your suggestions, I will go through the ROS tutorial.

And also looking forward to your example script!

Thanks, Jeremy

[supitalp]

Hi Jeremy, You can find a script here that will extract the events from a rosbag to a text file. You can use it as follows:

python extract_events_from_rosbag.py /path/to/sequence.bag --output_folder=/tmp --event_topic=/cam0/events --no-zip

Alternatively (especially if you need to produce a large number of datasets, where it would be impractical to store everything as rosbags before converting them to text), you can write your own publisher for ESIM to output the events to a text file instead of a rosbag. I just wrote a wiki page explaining how to do that.

Hope that helps.

[JeremyCJM]

Hi Henri,

Thanks a lot! I have tried your code just now, and it works well.

Thanks, Jeremy

[zcong17huang]

Hi Henri , Thanks a lot ! I meet the same problem as Jeremy's. rosrun dvs_renderer dvs_renderer events:=/cam0/events After running this, it got stuck. I don't know if it's running or finished.Now i know it's a loop and i should open a new terminal and run rosbag play /tmp/out.bag -l -r 0.1 (The new terminal thing you didn't mention in the your wiki page,so i suggest adding it) Then in another terminal run rqt_image_view /dvs_rendering and it works well !