RMSE Error on EuRoC Dataset

[YilmazKadir]

Hello, I am evaluating ROVIO algorithm in VIO mode. In the paper it is stated that in EuRoC MH_01 data they achieve 0.178m RMSE, and in V2_01 they achieve 0.064 m RMSE. Which alignment procedure are you using? I am using evo package and I achieve 0.213885 m RMSE for MH_01 and 0.103967 m RMSE for V2_01. Can you explain why I can not achieve the same results?

How I calculate RMSE: I run the algorithm in VIO mode and save the map. I open the maplab console, load the map and export the vertices using csv_export. I convert the vertices.csv file into TUM format. I use evo_ape euroc command with alignment (-a) and these are the results I get for MH_01:

APE w.r.t. translation part (m) (with SE(3) Umeyama alignment)

   max	0.987680
  mean	0.178013
median 0.162161
   min	0.006351
  rmse	0.213885
   sse	83.213396
   std	0.118567

[amichayfeldman]

I also got strange behavior. I didn't get the paper's results and in addition I got with ROVIOLI worse results than VIO mode in MH1 data.

[araujokth]

Have you tried running maplab from the pre-release-march-2018 branch https://github.com/ethz-asl/maplab/tree/pre_release_public/march-2018? As they state in the release notes you may be able to achieve better results https://github.com/ethz-asl/maplab/pull/55

Please see my updated comment on https://github.com/ethz-asl/maplab/issues/261#issuecomment-675296188, I think this solves your problem