How to run the train_ego_pred.py?

[Flawless1202]

I find the train_ego_pred.py use the data from HEVI_dataset/ego_motion. But I don't have the HEVI Dataset. Is there any way to train without HEVI Dataset? @MoonBlvd

[MoonBlvd]

@Flawless1202 If you would like to use your own dataset, just write your own dataloader by following the HEVIDataset in lib/utils/fol_dataloader.py. If you would like to use HEVI, I think Honda is planning to release the dataset to universities and academy in the recent 2 weeks. If you just want the ego motion data from the visual odometry, please find the data I process using ORBSLAM2 for train and validation. Each pickle file is a dictionary corresponds to one car's trajectory, and it has corresponding ego motions.

[Flawless1202]

Thanks for your reply. And I met another problem. I try to use ORB_SLAM2 to generate the ego motion data from the SA Dataset. But the videos have no camera calibration and distortion parameters. So how do you generate the ego motion data with ORB_SLAM2? @MoonBlvd

[MoonBlvd]

@Flawless1202 I didn't find intrinsic matrix neither, so I just used [fx fy cx cy] = [938.0666, 843.003, 643.298, 366.1718], which was for HEV-I. This will make the visual odometry not accurate for sure.

[Flawless1202]

@MoonBlvd Thanks for your reply. And these days I try to generate the ego motion with ORB_SLAM2, but the sequences is to short and often initialize failed. How do you to solve the problem? And when I try to generate the tracking result, it is not always continuous. Has this problem you ever met and how do you solve it? Look forwarding to your reply.

[MoonBlvd]

@Flawless1202 Yes, the ORB_SLAM2 won't work perfectly with video. I used linear interpolation which you can find in lib/utils/data_prep_utils.py

[Flawless1202]

@MoonBlvd Thanks. I also try to use linear interpolation and I also find it in lib/utils/data_prep_utils.py. But I am not really sure the data format which you called 'ORB_SLAM_FORMAT'. When I run the orb_slam2, I got the trajectory file format is 'timestamp x y z qx qy qz w', only 8 numbers in every line and can not use your data prepare scripts. What the data format you get from orb_slam2?

[SateeshKumar21]

hey @Flawless1202 did you manage to solve this problem?

[MoonBlvd]

Hey guys, I'm sorry if I have closed the issue by mistake. What I got from ORB_SLAM2 was text files. Each line of the text file was a flattend Tcws matrix. What I did here was just to load the lines and reshape it to a 4*4 matrix. @SateeshKumar21 @Flawless1202

[Flawless1202]

@MoonBlvd Thanks a lot. I have solved the problem, but the ORB_SLAM2 works too bad on my dataset. I don't think the odometry is accurate enough for the ego motion prediction task. Do you think the odometry is necessary?

[SateeshKumar21]

@Flawless1202 could you please explain how you solved this problem. Because right now I am also stuck at the same point that I have the 'timestamp x y z qx qy qz w' format and don't know how to convert it into the format used by @MoonBlvd.

[Flawless1202]

@SateeshKumar21 I think the load_vo function in lib/utils/data_prep_utils.py doesn't work well for the text file generated by ORB_SLAM2. So I use the formula to convert the quaternion to transformation matrix by myself.

[SateeshKumar21]

@MoonBlvd could you please specifically describe how you run ORB slam. I ran Orb-slam using the mono_TUM( https://github.com/raulmur/ORB_SLAM2/tree/master/Examples/Monocular/mono_tum) executable and I receive the text file of the format 'timestamp x y z qx qy qz w' which is not compatible with the scripts you provide.

[kimimasa-tamura-TRI]

@MoonBlvd Hi auther, first of all thank you for your great work!

As asked by others, could you please let us how to get the ORB result?

• SLAM.SaveKeyFrameTrajectoryTUM() -> outputs 'timestamp x y z qx qy qz qw', so it seems not. In addition, it output only key frames, not all the frames.
• SLAM.SaveTrajectoryTUM(); -> outputs all the frames, but not 4x4 matrix.
• Or did you add some codes at the end of System::SaveTrajectoryTUM() like below?

        cv::Mat Tcw_1d = Tcw.reshape(1);
        for (int i=0; i<16; i++)
        {
            f << Tcw_1d.at<float>(i) << " ";
        }
        f << endl;

Hey guys, I'm sorry if I have closed the issue by mistake. What I got from ORB_SLAM2 was text files. Each line of the text file was a flattend Tcws matrix. What I did here was just to load the lines and reshape it to a 4*4 matrix. @SateeshKumar21 @Flawless1202

[tianchenji]

@kimimasa-tamura-TRI Do you have any updates on this issue? I'm facing the same issue here.