code-iai
Add android support for API >= 27
What this PR does
- Update Build.cs to include Android target platform
- Built libbson using NDK for target API >= 27 and added it as
libbson-static-android-1.0.afor use with Android targets.
Using ROSIntegration on Android
To use with UE4 on Android, you'll need to
- setup: follow https://docs.unrealengine.com/en-US/Platforms/Mobile/Android/GettingStarted/index.html
- setup: follow https://docs.unrealengine.com/en-US/Platforms/Mobile/Android/Setup/AndroidStudio/index.html
- Additionally you'll need to set the
Project Settings -> Platforms -> Android SDK :: NDK API Levelto beandroid-27or higher since that's what I built libbson with.
Building libbson
The libbson static library was built from libbson with the following commands (in powershell on windows):
cmake -G "MinGW Makefiles" -DCMAKE_MAKE_PROGRAM="C:\Android\android-sdk\ndk\21.1.6352462\prebuilt\windows-x86_64\bin\make.exe" -DCMAKE_TOOLCHAIN_FILE="C:\Android\android-sdk\ndk\21.1.6352462\build\cmake\android.toolchain.cmake" -DANDROID_PLATFORM=android-27 ..
cmake --build .
The default android API UE4 supports (android-19 on 4.24) is too low for libbson to build, as it requires some things like stderr which were not added until later. I'm not exactly sure which API is the minimum, but I have selected API 27 for now.
Notes:
I wasn't sure which libbson version you used when building yours; I selected libbson 1.9.0.
Testing
I have tested that I am able to send messages from my android phone (Pixel 2XL) to an Ubuntu 18.04 VM on my Windows 10 desktop via WiFi.
bob@ubuntu ~/dev/roshi/src master ● rostopic list
/client_count
/clock
/nav_msgs
/rosout
/rosout_agg
/tf
/unreal_ros/spawn_objects
/unreal_ros/spawn_objects_array
e.g.
bob@ubuntu ~/dev/roshi/src master ● rostopic hz /nav_msgs
subscribed to [/nav_msgs]
average rate: 1.890
min: 0.529s max: 0.529s std dev: 0.00000s window: 2
average rate: 1.987
min: 0.458s max: 0.529s std dev: 0.03208s window: 4
average rate: 1.999
min: 0.458s max: 0.529s std dev: 0.02514s window: 6
average rate: 1.999
min: 0.458s max: 0.529s std dev: 0.02136s window: 8
average rate: 1.988
min: 0.458s max: 0.541s std dev: 0.02382s window: 10
average rate: 1.998
min: 0.458s max: 0.541s std dev: 0.02237s window: 12
bob@ubuntu ~/dev/roshi/src master ● rostopic echo /nav_msgs
header:
seq: 13
stamp:
secs: 119
nsecs: 11009216
frame_id: "odom"
poses:
-
header:
seq: 1
stamp:
secs: 0
nsecs: 505300402
frame_id: "odom"
pose:
position:
x: 0.00373643636703
y: 0.00014440715313
z: 0.26910340786
orientation:
x: 4.10079956055e-05
y: -0.00216591358185
z: -6.8187713623e-05
w: 0.999997735023
-
...
As a note - I'm currently getting an error when trying to receive image data into UE4 on Android:
D/UE4 : [2020.05.23-18.40.24:421][ 0]LogROS: Error: Error on BSON parse - Ignoring message
D/UE4 : [2020.05.23-18.40.24:450][ 0]LogTcpMessaging: Lost node '39D9096A4101D3A61ECDE58A0666B3C3' on connection '127.0.0.1:45026'...
D/UE4 : *** JavaEnvDestructor: 19744
F/DEBUG : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
F/DEBUG : Build fingerprint: 'google/taimen/taimen:10/QQ2A.200501.001.B3/6396602:user/release-keys'
F/DEBUG : Revision: 'rev_10'
F/DEBUG : ABI: 'arm'
F/DEBUG : Timestamp: 2020-05-23 13:40:24-0500
F/DEBUG : pid: 19636, tid: 19772, name: Thread-2 >>> com.permobil.MobileRoshiCPP <<<
F/DEBUG : uid: 10738
F/DEBUG : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
F/DEBUG : Cause: null pointer dereference
F/DEBUG : r0 00000000 r1 91134020 r2 009fff00 r3 00000000
F/DEBUG : r4 91134000 r5 00a00000 r6 bf041500 r7 00000000
F/DEBUG : r8 00001222 r9 94050680 r10 5cd01000 r11 91c31000
F/DEBUG : ip e5fd6c50 sp 91c30fb0 lr b5e9b63c pc e5f7f2fc
F/DEBUG :
F/DEBUG : backtrace:
F/DEBUG : #00 pc 0005d2fc /apex/com.android.runtime/lib/bionic/libc.so (__memcpy_base_a55+244) (BuildId: f9f8835945b295b609712530651a95ae)
F/DEBUG : #01 pc 05dd9638 /data/app/com.permobil.MobileRoshiCPP-DpNGnso6DXbErCjnsFy87g==/lib/arm/libUE4.so (FMallocBinned::Realloc(void*, unsigned int, unsigned int)+808) (BuildId: f904ed9abcb63cfd2aeca0dd5b0f21dab61aed06)
F/DEBUG : #02 pc 05e41940 /data/app/com.permobil.MobileRoshiCPP-DpNGnso6DXbErCjnsFy87g==/lib/arm/libUE4.so (FMallocPoisonProxy::Realloc(void*, unsigned int, unsigned int)+140) (BuildId: f904ed9abcb63cfd2aeca0dd5b0f21dab61aed06)
F/DEBUG : #03 pc 05dffc6c /data/app/com.permobil.MobileRoshiCPP-DpNGnso6DXbErCjnsFy87g==/lib/arm/libUE4.so (FMemory::Realloc(void*, unsigned int, unsigned int)+204) (BuildId: f904ed9abcb63cfd2aeca0dd5b0f21dab61aed06)
F/DEBUG : #04 pc 04ef33fc /data/app/com.permobil.MobileRoshiCPP-DpNGnso6DXbErCjnsFy87g==/lib/arm/libUE4.so (_ZN6TArrayIh22TSizedDefaultAllocatorILi32EEE10ResizeGrowEi+96) (BuildId: f904ed9abcb63cfd2aeca0dd5b0f21dab61aed06)
F/DEBUG : #05 pc 04ec8a88 /data/app/com.permobil.MobileRoshiCPP-DpNGnso6DXbErCjnsFy87g==/lib/arm/libUE4.so (TCPConnection::ReceiverThreadFunction()+1412) (BuildId: f904ed9abcb63cfd2aeca0dd5b0f21dab61aed06)
F/DEBUG : #06 pc 04efbc7c /data/app/com.permobil.MobileRoshiCPP-DpNGnso6DXbErCjnsFy87g==/lib/arm/libUE4.so (_ZNSt6__ndk114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEM13TCPConnectionFivEPS7_EEEEEPvSC_+60) (BuildId: f904ed9abcb63cfd2aeca0dd5b0f21dab61aed06)
F/DEBUG : #07 pc 000a5ea3 /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+20) (BuildId: f9f8835945b295b609712530651a95ae)
F/DEBUG : #08 pc 00060773 /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30) (BuildId: f9f8835945b295b609712530651a95ae)
At least I assume it's on the sensor_msgs::Image data (or possibly the nav_msgs::Path) since I do properly see this message from my code prior to the error:
D/UE4 : [2020.05.23-18.36.56:894][ 0]LogTemp: Warning: Got point cloud message!
Indicating that the PointCloud2 message was properly received (in UE4 on Android) and deserialized. I'm hoping it's unrelated to the libbson version, but it may be.
Update: I have confirmed that the error reported above occurs when parsing the nav_msgs::Path and not the sensor_msgs::PointCloud2. When only subscribing to the sensor_msgs::PointCloud2 the app works fine - no crashes and no errors or warnings in the log from the Android device.
I'll see if I can figure out why it's failing to parse the nav_msgs::Path message.
It also fails to parse the sensor_msgs::Image, but doesn't crash the program, instead getting a different error:
D/UE4 : [2020.05.23-18.51.03:895][ 0]LogROS: Error: Error on BSON parse - Ignoring message
followed by a lot of
D/UE4 : [2020.05.23-18.51.06:750][ 84]LogROS: Error: Failed to recv()
D/UE4 : [2020.05.23-18.51.06:751][ 84]LogROS: Error: Failed to recv()
D/UE4 : [2020.05.23-18.51.06:751][ 84]LogROS: Error: Failed to recv()
D/UE4 : [2020.05.23-18.51.06:752][ 84]LogROS: Error: Failed to recv()
D/UE4 : [2020.05.23-18.51.06:752][ 84]LogROS: Error: Failed to recv()
Further Update: I've confirmed that the same errors occur when receiving Path data / Image data on Windows - so it is not specific to the android bson library I built. I am able to successfully receive pointcloud data on Windows and Android, but unable to receive path or image data on either platform.
Update: I am able to receive on all platforms (Windows, Mac, Android - haven't tested linux):
sensor_msgs/PointCloud2tf2_msgs/TFMessagenav_msgs/Path
Windows and Mac:
sensor_msgs/Image- but only when it is a color image (32FC3), when it is a depth image (32FC1) the receipt fails with the same errors as above. Additionally, I've found that the ability to properly receive data is highly dependent on rosbridge state. If I have tried to send adepthimage through rosbridge before (without restarting rosbridge), then even new connections will be unable to receive color images properly (and in some cases may even fail to receive the messages above). The workaround for now is to ensure that rosbridge is always restarted when the other parts of my system are restarted.
Android:
- I am unable to receive any kind of image (color or depth) and it crashes the program when it is received.