Support interpolation methods of Accel and Gyro for IMU in SDK

[ruffsl]

Many conventional VIO pipelines intake internal measurements as synchronized pair of accelerometer and gyroscope streams. If the sample rates for these streams are unsynchronized, then a linear interpolation over time is often applied to unionize the two into a single measurement stream.

There's been a lot of duplicate work in downstream projects to facilitate this required abstraction: https://github.com/IntelRealSense/realsense-ros/pull/558 https://github.com/IntelRealSense/realsense-ros/pull/1010

Currently, such abstractions must be implemented for each new downstream library: https://github.com/intel/ros2_intel_realsense/issues/103 https://github.com/IntelRealSense/librealsense/issues/3921

As I'm at the cusp of porting this same filter from ROS1 to ROS2, I'd like to suggest we upstream this abstraction into the SDK so everyone can benefit from a complete and correct implementation for linear interpolation. Perhaps the sensor firmware itself could compute the filter, given the hardware always knows the SENSOR_TIMESTAMP for both a Accel and Gyro, circumventing failure modes from https://github.com/IntelRealSense/librealsense/issues/5710 , where such filtering on the host side could be inaccurate from imprecise clock domains.

cc @doronhi

[RealSenseCustomerSupport]

---

Hi @ruffsl

Thanks for posting this request. For our T265 tracking solution, the current way in which we handle the IMU (Accel and Gyro) works well for our SLAM algorithm. The sample rates in which we've chosen is optimized for our SLAM algo and are fixed. No plans to make changes to our sample rates in firmware.

Looking at the information in the various github links you've provided, doronhi has provided great information regarding how the SDK is handling the IMU data as well.

[ruffsl]

Hello @RealSenseCustomerSupport ,

The sample rates in which we've chosen is optimized for our SLAM algo and are fixed. No plans to make changes to our sample rates in firmware.

This request does not pertain to changing existing sample rates for the Accel and Gyro, only that an additional frame type be made available that interpolates the two into a single 6DoF motion frame.

Looking at the information in the various github links you've provided, doronhi has provided great information regarding how the SDK is handling the IMU data as well.

I think this kind of filtering or preprocessing is a core necessity for many user end domain applications, no different than any of the existing filters like the temporal interpolation of depth frames. Here is a case example of where downstream has had to re-implement this same feature for a second time:

https://github.com/intel/ros2_intel_realsense/issues/103

For our T265 tracking solution

At present, we happen to be using the D435i, so the firmware tracking for T265 is not applicable.

[RealSenseCustomerSupport]

---

Thanks for the update. Apologies for thinking this was pertaining to T265 product line. That was the context in which this thread was being viewed. We will look at it from the D435i standpoint.

[richard-elvira]

Hello @RealSenseCustomerSupport, I see that the SDK allows to interpolate the Inertial data to obtain sync measures from the gyroscope and accelerometer but I only find information in ROS and I need to obtain the synchronized IMU information without ROS.

I am working in a C++ project with RealSense T265. The code obtains the information by callbacks but the gyroscope and accelerometer are not synchronized. There is a way to apply the interpolate solution in that scenario.

Thanks.

[MartyG-RealSense]

Hi @ruffsl I am now handling this case and would like to provide you with an update.

There is an active Intel feature request relating to the requested feature of supporting interpolation methods of Accel and Gyro for the IMU in the RealSense SDK. There is no progress to report regarding this request at the time of writing this though.