ORB_SLAM3
ORB_SLAM3 copied to clipboard
UZ-SLAMLab
Running Mono Inertial datasets collected from ANDROID smartphone [ON camera-IMU transofrmation and its IMPORTANCE]
Hi there,
I am trying to run sequences taken with my smartphone (Huawei P20 light) but without success so far. The frames goes by in the viewer but the system is stuck in "trying to initialize" and no features are find, map is ofc also white empty. I don't even see any green struggling lines which usually are good sign of ORBSLAM working for matching some points but not even that.
I run the same exact sequences in Visual-only monocular and they work just fine but when I call mono_inertial_tum_vi with IMU datas it just won't elaborate them. I am sure that the problem is in the .yaml file since if I use run the dataset with the .yaml file from Euroc ( ./Examples/Monocular-Inertial/EuRoC.yaml ), the system is sometimes able to track some points or at least I can see it working.
I took the datas with marslogger app which gives properly synchronized datas (https://github.com/OSUPCVLab/mobile-ar-sensor-logger).
Now, I am pretty sure that my intrinsic camera parameters are fine since they work well with visual only executables.
That only lives the problem in the IMU-camera transformation matrix and eventually the IMU noises but I guess this is unlike to happen. (I am using standards IMU noises for a smatphone gyro and acc, found them online) I have tried such a matrix considering that in smartphone the distance between camera and sensors could have been ignored but apparently not. [0, -1, 0, 0; -1, 0, 0, 0; 0, 0, -1, 0; 0, 0, 0, 1]
Is it possible that for ORBSLAM to run properly a very precise tranformation matrix is needed? Do I need to calibrate the IMU-camera system to find out? Or am I missing something.
I'm also struggling with this issue. I have tested all possible TBCs and the issue remains unsolved.
Hello, eventually I solved performing the calibration myself with Kalibr (https://www.youtube.com/watch?app=desktop&v=puNXsnrYWTY) this allowed me to find the camera-imu transformation matrix. It requires some time and work tho
This is the matrix that I found. Smartphone is a Huawei P20 lite. It works fine enough to detect some features in my case, but it still struggles some times
Tbc: !!opencv-matrix rows: 4 cols: 4 dt: f data: [0.9972997819302696, 0.0734334153322293, 0.0008236956224966672, -0.003837675482538145, 0.07334067505978509, -0.9964923296747585, 0.04030114491109185, 0.05140614751884478, 0.0037802570824251765, -0.040131912638374456, -0.9991872393322356, 0.2824587673685369, 0.0, 0.0, 0.0, 1.0]
Hi, yes, I simply recorded a rosbag of visual inertial data following all the instruction from the video that i linked above. I am aware it might not be the most precise matrix since it was hard to perform a precise calibration. For anyone who is willing to do it by himself and don't know how to record visual inertial datas I advice: https://github.com/OSUPCVLab/mobile-ar-sensor-logger
I will also post this issue I came across using Kalibr https://github.com/ethz-asl/kalibr/issues/146.
Hi GabHoo, I recently ran the Mono_Inertial with a android phone, while I found that it is very hard to initialization, and the error "Failed to track local map" occurred lot of times. I'm wondering if it is related to the calibration, or something else?
Hi GabHoo, I recently ran the Mono_Inertial with a android phone, while I found that it is very hard to initialization, and the error "Failed to track local map" occurred lot of times. I'm wondering if it is related to the calibration, or something else? It is hard for me to say, sorry. but yes, as I said, performing a precise Calibration is hard and therefore a bad Matrix could be the cause. Also android IMU sensors might not be the best in quality so it is hard in first place :(
Hi @GabHoo @bereze @hiddennasty I am about to try adapting ORB-SLAM3 into an Android app. Have you succeeded? I found this repository where someone seems to have made it using only the monocular : https://github.com/ChallengeHe/ORB_SLAM3_Android
