UZ-SLAMLab
orbslam3 for android
Hi,
Did any one any one cross compile orbslam 3 for android yet? Please let me know.
Thank you, Varun
Hi, I can compile it with Android Studio3.5.1 but still have lots of bugs after that. If you are still interested in this project, please send me an email. Thanks. [email protected]
compile to android should be fairly easy, but the keep the same performance as on PC is pretty challenge.
Hi,
A bit of an update on this.
I've been experimenting with compiling ORB-SLAM3 on Android (in addition to emscripten, see https://github.com/raulmur/ORB_SLAM2/issues/264), see here:
https://github.com/nickw1/orb-slam3-android-expts/
The code is based on the modified emscripten version, as I had already commented out a lot of the UI code so figured it would be a good place to start.
It compiles and runs, but crashes, see the stack trace in the repo above. Oddly it seems to be crashing with a SIGILL (illegal instruction) in C++ standard library code called by the DBoW2::Vector() constructor.
Wondering if there's anyone on here who are NDK/Android C++ experts (I'm just beginning with NDK, to be honest) who might be able to suggest what's happening? Thanks.
To update, on a real device (specifically, a Pixel 3 running Android 12) it doesn't seem to crash on startup (i.e. all code in ORB_SLAM3::System constructor completes without crash).
It compiles and runs, but crashes, see the stack trace in the repo above. Oddly it seems to be crashing with a SIGILL (illegal instruction) in C++ standard library code called by the
DBoW2::Vector()constructor.
I don't find a DBoW2::Vector(), but there is a DBoW2::BowVector(), but is is a default contructor for the std::map<unsigned int, double> superclass.
Hard to say what is causing the crash.
Yes, sorry, that was my mistake, I meant DBoW2::BowVector().
Maybe (as suggested by @carlodek on the Emscripten issue) I will use standard Android camera APIs to get the current frame (rather than OpenCV for Android); it's possible, I guess, that having two different copies of OpenCV on the front and back end might be causing an issue.
I have ported the ORB SLAM 3 to android platform. Plain visual mode works now. I am working on the Monocular-Inertial mode.
@faywong, those are great news!
Do you have a video? Can we see how fast/slow it runs in mapping and in localization?
Status update on 2/15/2023:
I have brought up the whole ORB SLAM3 mono-interial mode on Android Phone. but the imu data regularly got reset with error:
mnFirstFrameId = 13
mnInitialFrameId = 0
2 Frames set to lost
First KF:2; Map init KF:0
New Map created with 367 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
@faywong has the accel been calibrated?
Both camera and the IMU have been calibrated. In Q4 of this year, i will back to address this issue.
Hello,
Are you able to provide a github/instructions on your work for this? I also want to compile this for Android.
@jagauthier I got it compiling on Android last year, see repo above, though it's currently not doing anything, as such (crashes when run on emulator, runs without crashing on Pixel 3/Android 12 but haven't got it doing anything yet). Nonetheless the repo might provide a useful starting point, it includes some instructions.
Hi Nick,
Thank you very much, I will give it a try.
Varun
On Wed, Aug 16, 2023 at 8:56 AM Nick Whitelegg @.***> wrote:
@jagauthier https://github.com/jagauthier I got it compiling on Android last year, see repo above, though it's currently not working, as such (crashes when run). Nonetheless the repo might provide a useful starting point, it includes some instructions.
— Reply to this email directly, view it on GitHub https://github.com/UZ-SLAMLab/ORB_SLAM3/issues/77#issuecomment-1680873339, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKJ3G2VSMAWWRUH6Q3K5STXVTUSLANCNFSM4QGVA55A . You are receiving this because you authored the thread.Message ID: @.***>
I guess many people are trying to implement this facinating project on android and isn't succeeding or have low fps.
Will it be interesting to avoid the JNI by making a full ORB SLAM3 java based? Maybe this would result in better performances
@MasterHansCoding
I don't know any mobile orbslam implementation working at a decent speed.
JNI is the native interface. There's nothing faster than native. A full Java orbslam will be much slower, and the garbage collector will interfere.
They managed to use ORB SLAM 2 technique on smartphone + computer server at 35fps : https://ieeexplore.ieee.org/abstract/document/9155438 The computer server is doing some CPU expansive tasks of the ORB SLAM pipeline. This article is available in google scholar.
@MasterHansCoding
Great paper!
I am actually working in a visual slam partition. In 2022 we succeded on running a preprocessor on an old mobile and the rest on a backend. This path is an alternative to have vSLAM running entirely con a phone.
The main difference with the paper, we aim to use any phone, not only high end ones.
Btw, the paper doesn't mention Java but it mentions OpenCV on the edge device, meaning at least feature extraction is done native. May be a way to identify a set of performant functions to be compiled into native and interfaced with JNI, while the rest can be done in Java.
They managed to use ORB SLAM 2 technique on smartphone + computer server at 35fps : https://ieeexplore.ieee.org/abstract/document/9155438 The computer server is doing some CPU expansive tasks of the ORB SLAM pipeline. This article is available in google scholar.
Great info, unfortunately that work is not open-sourced, but I just found another similar project (and paper) which is. Github: https://github.com/droneslab/edgeslam Paper: Edge-SLAM: Edge-Assisted Visual Simultaneous Localization and Mapping ( https://dl.acm.org/doi/pdf/10.1145/3561972 )
ARCore managed to run VI-SLAM on mobile device without work done on external computer and got 30-50 fps. I wish I could know how they managed ...
I successfully ported orb3 from Ubuntu to Android, and both Ubuntu and Android are installed on the same development board. I found that orb3 on Android was much slower to run than orb3 on Ubuntu, even on the same development board. The specific performance is that the front-end running time difference is not big,30fps. The back-end running time Android is 3 times slower than Ubuntu, I don't know if it is the problem of g2o.If any friends are interested in this, welcome to communicate with me.email [email protected].
