rtabmap icon indicating copy to clipboard operation
rtabmap copied to clipboard

Question about determinism

Open schmidtp1 opened this issue 2 years ago • 2 comments

Hi, I was interested in how far the output is expected to match between runs. I was hoping that with the noEventExample (on stereo_20Hz) it would be very close but observed quite some differences. Would you be able to provide any insight what could (mostly) cause the deviations and how much to expect? Thank you!

test ATE: 0.090976

schmidtp1 avatar Jul 20 '22 00:07 schmidtp1

Thinking more about it, there is the RANSAC used in this case. I will look more into it (e.g. if the random seed can be set). Closing. (Please let me know if you can provide any insights/hints.)

schmidtp1 avatar Jul 20 '22 02:07 schmidtp1

I believe the random seed is fixed by default. (Anyway I set a constant random seed for opencv.) But there is still a lot of variance. Can you provide any insight?

schmidtp1 avatar Jul 20 '22 19:07 schmidtp1

Some insight here: http://official-rtab-map-forum.206.s1.nabble.com/Repeatability-over-same-data-tp9130p9201.html

I tried four times the no events example with those changes:

diff --git a/examples/NoEventsExample/main.cpp b/examples/NoEventsExample/main.cpp
index 87fb8fed..04ceb97a 100644
--- a/examples/NoEventsExample/main.cpp
+++ b/examples/NoEventsExample/main.cpp
@@ -119,6 +120,7 @@ int main(int argc, char * argv[])
                        if(cameraIteration++ % odomUpdate == 0)
                        {
                                OdometryInfo info;
+                               data.setStamp(double(cameraIteration)/double(cameraRate));
                                Transform pose = odom->process(data, &info);
 
                                if(odometryIteration++ % mapUpdate == 0)
@@ -153,6 +155,7 @@ int main(int argc, char * argv[])
                        printf("Processed all frames\n");
                        app.exec();
                }
+               rtabmap.exportPoses("poses.txt", true, true, 10);
        }
        else
        {

Note that I fixed in commit above the exportPoses with format 10 (RGB-D dataset format without orientation change). Also re-stamped the data for the evaluate_ate.py tool (to have same stamps between runs). Then increased Map Update Rate to 2 to get more poses:

$ rtabmap-noEventsExample 20 2 5 stereo_20Hz stereo_20Hz stereo_20Hz/left stereo_20Hz/right

Results: Screenshot from 2022-11-26 12-40-41

ATE:

  • B->A = 0.021953
  • C->A = 0.019460
  • D->A = 0.024855
  • E->A = 0.022301

Mean=0.022142, std=0.0022071

matlabbe avatar Nov 26 '22 20:11 matlabbe

Thank you, that's good news! Thank you for looking into it! Closing.

schmidtp1 avatar Nov 28 '22 20:11 schmidtp1