The event visualization of the hsergb dataset

[danqu130]

Thanks to open source for this amazing work! I have two questions about the hsergb dataset.

python test_loader.py --dataset_root data/hsergb --dataset_type close --sequence spinning_umbrella --sample_index 400

python test_loader.py --dataset_root data/hsergb --dataset_type close --sequence fountain_schaffhauserplatz_02 --sample_index 420

From the above two examples, I found that some events are not aligned with the image, and some grid locations are missing events. Is this normal? And why are the coordinates (x, y) in the event data decimal value rather than integer value? Is this the raw data collected by the event camera, or due to the interpolation during dual camera calibration?

[HaoDot]

same questions！ Chances are that the reason of the coordinates is not int is the calibration between event camera and RGB camera. So is the spatial dis-alignment. The close part of data can't be aligned due to the calibration. However, in my opinion, this problem can be alleviated by using far part of data.

[danielgehrig18]

Hi guys, the reason why events have decimal values is that we align the event coordinates with the frames via a tomography, so

x_event_new = H x_event_old

Where H has floating point values, x_event_old, are the integer value event coordinates. To have maximal accuracy we need to retain the decimal value, and we also use it when constructing the network input. The visualizer round the event coordinates for visualization purposes only! For this reason, some pixes are missed, since no event coordinate (when rounded) maps to that coordinate. Let me know if you have found a better way of visualizing which takes into account floating point coordinates.

[danqu130]

Hi guys, the reason why events have decimal values is that we align the event coordinates with the frames via a tomography, so

x_event_new = H x_event_old

Where H has floating point values, x_event_old, are the integer value event coordinates. To have maximal accuracy we need to retain the decimal value, and we also use it when constructing the network input. The visualizer round the event coordinates for visualization purposes only! For this reason, some pixes are missed, since no event coordinate (when rounded) maps to that coordinate. Let me know if you have found a better way of visualizing which takes into account floating point coordinates.

@danielgehrig18 Thank you for your answer. Is it more appropriate to warp the image to the event camera coordinate system to ensure that the raw event data is not changed? However, the events in the above data are not aligned well with the image. Is it normal?

[danielgehrig18]

This could also be done. However, since we anyway undistort and rectify both the events and frames, we decided to do it. At this point it should not make a big difference. Can you clarify what you mean with However, the events in the above data are not aligned well with the image. Is it normal?

On Wed, Mar 2, 2022 at 9:06 AM danquxunhuan @.***> wrote:

Hi guys, the reason why events have decimal values is that we align the event coordinates with the frames via a tomography, so

x_event_new = H x_event_old

Where H has floating point values, x_event_old, are the integer value event coordinates. To have maximal accuracy we need to retain the decimal value, and we also use it when constructing the network input. The visualizer round the event coordinates for visualization purposes only! For this reason, some pixes are missed, since no event coordinate (when rounded) maps to that coordinate. Let me know if you have found a better way of visualizing which takes into account floating point coordinates.

@danielgehrig18 https://github.com/danielgehrig18 Thank you for your answer. Is it more appropriate to warp the image to the event camera coordinate system to ensure that the raw event data is not changed? However, the events in the above data are not aligned well with the image. Is it normal?

— Reply to this email directly, view it on GitHub https://github.com/uzh-rpg/rpg_timelens/issues/28#issuecomment-1056517301, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFBHKKHVA76VE6SLVIJHXFDU54OORANCNFSM5IQNYFNQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[danqu130]

This could also be done. However, since we anyway undistort and rectify both the events and frames, we decided to do it. At this point it should not make a big difference. Can you clarify what you mean with However, the events in the above data are not aligned well with the image. Is it normal? … On Wed, Mar 2, 2022 at 9:06 AM danquxunhuan @.> wrote: Hi guys, the reason why events have decimal values is that we align the event coordinates with the frames via a tomography, so x_event_new = H x_event_old Where H has floating point values, x_event_old, are the integer value event coordinates. To have maximal accuracy we need to retain the decimal value, and we also use it when constructing the network input. The visualizer round the event coordinates for visualization purposes only! For this reason, some pixes are missed, since no event coordinate (when rounded) maps to that coordinate. Let me know if you have found a better way of visualizing which takes into account floating point coordinates. @danielgehrig18 https://github.com/danielgehrig18 Thank you for your answer. Is it more appropriate to warp the image to the event camera coordinate system to ensure that the raw event data is not changed? However, the events in the above data are not aligned well with the image. Is it normal? — Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFBHKKHVA76VE6SLVIJHXFDU54OORANCNFSM5IQNYFNQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.Message ID: @.>

For example, it seems that the edge of the letter part in the book of the second picture is not aligned with the events.