Dynamic object removal: real-time performance and odometry loss

[yongyun-iastate]

Hello, thank you for your great work. I am currently removing dynamic objects using an approach similar to DOTMASK, but I am wondering if this could run in real time. My filtered depth runs at 4 FPS, and /rtabmap/odom runs at 1 Hz, which doesn’t really seem like real time, and the odometry is lost frequently. My questions are:

1. can we use this approach in real-time
2. which parameter should I adjust it?

Thank you in advance.

[matlabbe]

What is the latency on this 4 FPS?

#ros1
rostopic delay /depth 
#ros2
ros2 topic delay /depth

And what is the frequency of the RGB and original depth topics and their latency?

Check what is the processing time to update /rtabmap/odom in the logs. That could give you the theoretical time that you can reach with that computer for the image resolution used. For example, if the processing time is under 100 ms, then you should be able to achieve at least 10 Hz. If it is the case and it runs at 1 Hz, then maybe rtabmap has difficulty to synchronize the RGB and depth frames together. For example, if RGB and original depth are published at 30 Hz with 60 ms delay, and your filtered depth is published at 4 Hz with 1.5 sec delay, the sync_queue_size and/or topic_queue_size would need to be at least 60 (default 10) to synchronize ALL filtered depth images and get at least 4 Hz instead of 1 Hz.

[yongyun-iastate]

Thank you for answering my question! The latency of the filtered depth is 200 ms and I solved the problem. The problem was there was a delay when transforming camera frame to map frame because I was trying to exact sync.

Beyond that, can I ask why the mean SLAM processing time doubled compared to usual? It was 50 ms normally, but when I used the filtered depth it became 100 ms. I am trying to compare this with the ORB-SLAM3 version of the dynamic object removal algorithm from other people. However, the mean tracking time of ORB-SLAM3 was around 30 ms when it comes to only slam processing, even when they removed dynamic objects which makes hard to compare entire mean processing time.

Then, should I compare mean rtabmap odom processing time with orb slam3 mean tracking time? Since it seems like orb slam does not do dense mapping as compared to rtabmap.

[matlabbe]

I think ORB-SLAM3 would report only odometry tracking time, which would be visual odometry time for rtabmap. Note that ORB-SLAM3 global bundle adjustment after a loop closure would take significantly more time than the graph optimization done in rtabmap after loop closure.