depthai-core icon indicating copy to clipboard operation
depthai-core copied to clipboard

Published 20 hours ago • verified publisher icon luxonis

imu data not consistent with earth gravity at rest (needs zgo calibration?)

Open diablodale opened this issue 3 years ago • 12 comments

Data returned from an IMU node is not consistent with Earth gravity of 9.81 m/s^2 on the Y-axis. My OAK-D instead returns a vector magnitude value 10.20799. And the gravity component changes between X and Y axes. Please reference the IMU Datasheet provided by Luxonis https://www.ceva-dsp.com/wp-content/uploads/2019/10/BNO080_085-Datasheet.pdf

Setup

  • OAK-D via BW1098OBC
  • depthai-core v2.13.3 + xlink move/thread fixes
  • Microsoft Windows [Version 10.0.19044.1415]

Repro

  1. Place your OAK-D on a flat stable non-moving surface with the RGB camera facing you and the USB port towards Earth center.
  2. Run the example imu_gyroscope_accelerometer.exe

Result

Accelerometer timestamp: 3867 ms
Accelerometer [m/s^2]: x: -0.306 y: 10.190 z: -0.498
Gyroscope timestamp: 3868 ms
Gyroscope [rad/s]: x: 0.009 y: -0.007 z: 0.007
Accelerometer timestamp: 3885 ms
Accelerometer [m/s^2]: x: -0.345 y: 10.190 z: -0.421
Gyroscope timestamp: 3886 ms
Gyroscope [rad/s]: x: 0.007 y: -0.006 z: 0.001
Accelerometer timestamp: 3901 ms
Accelerometer [m/s^2]: x: -0.345 y: 10.190 z: -0.498
...

The magnitude of the accelerometer data above is 10.20799 which is 4% larger than expected. The IMU datasheets claims an accuracy of 0.3 m/s^2. 10.20799 - 9.81 = 0.39799 making it outside the datasheet's accuracy claim and 5% larger than expected 9.81.

This unexpected accelerometer data is seen in all enums ACCELEROMETER, ACCELEROMETER_RAW, and GRAVITY In a separate app I wrote, I see the same errors across different outputs...

ACCELEROMETER       -10.109375    -0.34375    -0.460938
ACCELEROMETER_RAW    -0.344765    10.151415   -0.459687
GRAVITY             -10.1875      -0.34375    -0.492188

In addition, the gravity component (very likely the 10.1 value) changes between the X and Y axes depending on what output is chosen with enum class dai::IMUSensor. This is unexpected and not in alignment with the IMU datasheet section 2 which specifies a fixed coordinate system. This would then make it impossible for the gravity component to change axes.

Expected

At rest, the normal of the x,y,z vector from the accelerometer should be ~9.81.

I would expect the coordinate system for acceleration and gravity to be shared as documented in IMU datasheet section 2. Since gravity is derived from the raw accelerometer. Meaning I would e:xpect: accelerometer_linear + gravity = accelerometer. But that is not possible given the actual data being output via depthai APIs.

It is also unusual the gravity component changes +/-, but maybe the documentation is lax in that. Yes, I can negate and flip order...but it is suspicious that the "10" component is changing its coordinate system. It makes me suspect math error, or pointer offset error, struct member name/misuse, etc.

Notes

Is the 5% error due to section 3.2.1 ZGO errors? If so, then we need a mechanism to "dynamic calibrate" datasheet section 3 the accelerometer -as distinct- from extrinsic calibration of https://github.com/luxonis/depthai-core/issues/155

Or has there been a "Static calibration" datasheet section 3 burned into the IMU already?

diablodale avatar Dec 31 '21 01:12 diablodale

I would expect the coordinate system for acceleration and gravity to be shared as documented in IMU datasheet section 2. Since gravity is derived from the raw accelerometer. Meaning I would e:xpect: accelerometer_linear + gravity = accelerometer. But that is not possible given the actual data being output via depthai APIs.

This indeed should be the case, as a matter of fact, I tested on several boards, and the coordinate system is the same for gravity/accelerometer. Here is the test script. Can you set DEPTHAI_LEVEL=info and confirm that the IMU FW version is the following:

[IMU(0)] [info] IMU product ID:
[IMU(0)] [info] Part 10004148 : Version 3.9.7 Build 224
[IMU(0)] [info] Part 10003606 : Version 1.7.3 Build 337
[IMU(0)] [info] Part 10004135 : Version 5.5.1 Build 159
[IMU(0)] [info] Part 10004149 : Version 5.1.11 Build 182

Currently we don't calibrate IMU, so no extrinsic calibration or static/dynamic calibration is performed therefore there are differences between boards/accuracy. We expect to add IMU calibration next year in Q1.

SzabolcsGergely avatar Dec 31 '21 04:12 SzabolcsGergely

Hi. I see unexpected results.

  1. installed current v3.0.7 using https://docs.luxonis.com/en/latest/#demo-script
  2. verified demo py app works
  3. saved your linked test script to file
  4. ran that script
// dai.IMUSensor.GRAVITY
Accelerometer timestamp: 1388.323 ms
Accelerometer [m/s^2]: x: -10.171875 y: -0.625000 z: 0.570312
Total: 10.20700359377233

// dai.IMUSensor.ACCELEROMETER
Accelerometer timestamp: 688.716 ms
Accelerometer [m/s^2]: x: -10.109375 y: -0.613281 z: 0.613281
Total: 10.146511256280561

// dai.IMUSensor.ACCELEROMETER_RAW
Accelerometer timestamp: 685.052 ms
Accelerometer [m/s^2]: x: -0.651223 y: 10.113108 z: 0.574608
Total: 10.15033068502249

The "10" component is changing axes between X and Y. This is the same unexpected results I got with my C++ in OP.

Info debug output shows significantly different versions that yours...

[14442C10C1A1C6D200] [107.872] [system] [info] Memory Usage - DDR: 0.12 / 358.55 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 6.72 / 78.59 MiB, LeonRT Heap: 2.88 / 23.84 MiB
[14442C10C1A1C6D200] [107.872] [system] [info] Temperatures - Average: 30.01 ┬░C, CSS: 31.77 ┬░C, MSS 29.83 ┬░C, UPA: 28.62 ┬░C, DSS: 29.83 ┬░C
[14442C10C1A1C6D200] [107.872] [system] [info] Cpu Usage - LeonOS 11.92%, LeonRT: 2.45%
[14442C10C1A1C6D200] [107.882] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '16384'B
[14442C10C1A1C6D200] [108.133] [IMU(0)] [Accelerometer timestamp: 0.000 ms
Accelerometer [m/s^2]: x: -0.612916 y: 10.113108 z: 0.536301
Total: 10.145847994191397
info] IMU product ID:
[14442C10C1A1C6D200] [108.133] [IMU(0)] [info] Part 10004148 : Version 3.2.13 Build 6

[14442C10C1A1C6D200] [108.133] [IMU(0)] [info] Part 10003606 : Version 1.2.4 Build 230

[14442C10C1A1C6D200] [108.133] [IMU(0)] [info] Part 10003254 : Version 4.4.3 Build 485

[14442C10C1A1C6D200] [108.133] [IMU(0)] [info] Part 10003171 : Version 4.2.10 Build 548

diablodale avatar Dec 31 '21 17:12 diablodale

@diablodale you have a device that was shipped before factory calibration, which has older IMU FW. We don't have access to the IMU FW, this issue looks like got fixed with a FW update. Please checkout the following branch: https://github.com/luxonis/depthai/tree/UI-test-tools-Production Run install_requirements.py then oak-d.sh (python3 depthai_demo.py -s left right previewout meta_d2h jpegout -monor 400 -monof 10 -rgbf 10 -tm OAK_D_test). It will update IMU firmware to the latest.

SzabolcsGergely avatar Dec 31 '21 20:12 SzabolcsGergely

Yes. This was discovered and led to me using the cali tool for depth<->color several weeks ago.

It is possible now or in the future for customers to have IMUs that don't meet the requirements of depthai-core. My OAK-D is an example. Could depthai-core do the check itself on IMU startup? Or can it expose the firmware versions so my app can do the check?

dai::EepromData doesn't show any obvious way to check IMU firmware.

diablodale avatar Dec 31 '21 20:12 diablodale

Could depthai-core do the check itself on IMU startup

Yes, will add it.

SzabolcsGergely avatar Dec 31 '21 22:12 SzabolcsGergely

Got it. I also understand you have a queue of dev work. I will delay updating my IMU firmware as long as I can. Hopefully that will be enough time for you to add code and then I can verify that my app correctly handles the error/exception on IMU startup. Happy new year! 🎉🎆

diablodale avatar Dec 31 '21 22:12 diablodale

No improvement with updated IMU firmware. My OAK-D IMU accelerometer outputs are still unexpected/inconsistent

  1. I followed your IMU firmware update procedure. I got the green dot after it updated.
  2. I then ran your imu_test script. It now reports exact the same firmware as you list above. 
    [14442C10C1A1C6D200] [96.995] [IMU(0)] [Accelerometer timestamp: 0.000 ms
Accelerometer [m/s^2]: x: -10.171875 y: -0.257812 z: -0.523438
infoTotal: 10.188596327129046
] IMU product ID:
[14442C10C1A1C6D200] [96.995] [IMU(0)] [info] Part 10004148 : Version 3.9.7 Build 224
[14442C10C1A1C6D200] [96.995] [IMU(0)] [info] Part 10003606 : Version 1.7.3 Build 337
[14442C10C1A1C6D200] [96.996] [IMU(0)] [info] Part 10004135 : Version 5.5.1 Build 159
[14442C10C1A1C6D200] [96.996] [IMU(0)] [info] Part 10004149 : Version 5.1.11 Build 182
  3. I then ran the imu_test script with the three different IMU outputs: accelerometer, accelerometer_raw, and gravity

Result

No change. The outputs continue to be unexpected. I get the same signs and values as in the OP. Here are outputs from your test script and changing line 18-20 to get the different outputs My early production OAK-D is somewhat level sitting on a table.

// ACCELEROMETER
Accelerometer timestamp: 515.076 ms
Accelerometer [m/s^2]: x: -10.156250 y: -0.296875 z: -0.468750
Total: 10.171394957950703
Accelerometer timestamp: 530.792 ms
Accelerometer [m/s^2]: x: -10.269531 y: -0.269531 z: -0.488281
Total: 10.284665175325577

// ACCELEROMETER_RAW
Accelerometer timestamp: 506.852 ms
Accelerometer [m/s^2]: x: -0.287304 y: 10.199299 z: -0.497994
Total: 10.215490143768207
Accelerometer timestamp: 522.568 ms
Accelerometer [m/s^2]: x: -0.287304 y: 10.160992 z: -0.507571
Total: 10.177717005808487

// GRAVITY
Accelerometer timestamp: 184.832 ms
Accelerometer [m/s^2]: x: -10.210938 y: -0.273438 z: -0.523438
Total: 10.228000758296742
Accelerometer timestamp: 204.582 ms
Accelerometer [m/s^2]: x: -10.210938 y: -0.273438 z: -0.523438
Total: 10.228000758296742

diablodale avatar Feb 01 '22 21:02 diablodale

@diablodale check https://github.com/luxonis/depthai-core/pull/375

Calibration will be adressed later.

SzabolcsGergely avatar Feb 12 '22 02:02 SzabolcsGergely

@diablodale I would agree with you here that I observe the same thing on quite a few Series 2 units I have, installed with BNO085. The gravity vector is often too large at rest.

chengguizi avatar Jun 01 '22 02:06 chengguizi

@szabi-luxonis I tested again with depthai-core v2.15.4 (it includes pr #375) The axis are now aligned similar/expected.

// ACCELEROMETER_RAW
Accelerometer [m/s^2]: x: -10.142 y: -0.536 z: 0.871
Accelerometer [m/s^2]: x: -10.104 y: -0.546 z: 0.910

// ACCELEROMETER
Accelerometer [m/s^2]: x: -10.000 y: -0.555 z: 0.918
Accelerometer [m/s^2]: x: -10.090 y: -0.480 z: 0.883

// GRAVITY
Accelerometer [m/s^2]: x: -10.000 y: -0.555 z: 0.922
Accelerometer [m/s^2]: x: -10.000 y: -0.547 z: 0.922

I consider the axis portion of this issue as resolved. 👍 I understand the calibration is to be addressed later.

diablodale avatar Jun 01 '22 15:06 diablodale

hi @szabi-luxonis, how can an app using depthai-core v2.20.2 know that the IMU needs a firmware update? https://github.com/luxonis/depthai-core/issues/744

diablodale avatar Mar 02 '23 10:03 diablodale

The accelerometer data on my OAK-D-PRO-POE continues to be wrong. https://github.com/luxonis/depthai-core/pull/375 did not fix this. Below data is with both sensors level using depthai-core v2.25.0 example imygyroaccel.exe adapted to show accel and accel_raw.

OAK-D-PRO-POE

[18443010318EF50800] [192.168.2.23] [5.323] [IMU(0)] [info] IMU product ID:
[18443010318EF50800] [192.168.2.23] [5.323] [IMU(0)] [info] Part 10004563 : Version 3.9.9 Build 2
[18443010318EF50800] [192.168.2.23] [5.323] [IMU(0)] [info] Part 10003606 : Version 1.8.0 Build 338
[18443010318EF50800] [192.168.2.23] [5.323] [IMU(0)] [info] Part 10004135 : Version 5.5.3 Build 162
[18443010318EF50800] [192.168.2.23] [5.323] [IMU(0)] [info] Part 10004149 : Version 5.1.12 Build 183
Accelerometer     [m/s^2]: x: 11.574 y: 0.191 z: -0.238 
Accelerometer     [m/s^2]: x: 11.539 y: 0.152 z: -0.258 
Accelerometer     [m/s^2]: x: 11.586 y: 0.172 z: -0.305 
Accelerometer     [m/s^2]: x: 11.539 y: 0.191 z: -0.289 
Accelerometer     [m/s^2]: x: 11.551 y: 0.172 z: -0.219 
Accelerometer     [m/s^2]: x: 11.551 y: 0.172 z: -0.305 
...
Accelerometer Raw [m/s^2]: x: 11.530 y: 0.172 z: -0.278 
Accelerometer Raw [m/s^2]: x: 11.550 y: 0.182 z: -0.249 
Accelerometer Raw [m/s^2]: x: 11.588 y: 0.211 z: -0.259 
Accelerometer Raw [m/s^2]: x: 11.559 y: 0.201 z: -0.239 
Accelerometer Raw [m/s^2]: x: 11.588 y: 0.172 z: -0.249 
Accelerometer Raw [m/s^2]: x: 11.588 y: 0.230 z: -0.230 
Accelerometer Raw [m/s^2]: x: 11.569 y: 0.163 z: -0.230

OAK-D

[14442C10C1A1C6D200] [2.12] [0.911] [IMU(0)] [info] IMU product ID:
[14442C10C1A1C6D200] [2.12] [0.911] [IMU(0)] [info] Part 10004563 : Version 3.9.9 Build 2
[14442C10C1A1C6D200] [2.12] [0.911] [IMU(0)] [info] Part 10003606 : Version 1.8.0 Build 338
[14442C10C1A1C6D200] [2.12] [0.911] [IMU(0)] [info] Part 10004135 : Version 5.5.3 Build 162
[14442C10C1A1C6D200] [2.12] [0.911] [IMU(0)] [info] Part 10004149 : Version 5.1.12 Build 183
Accelerometer     [m/s^2]: x: -10.008 y: 0.066 z: 0.785 
Accelerometer     [m/s^2]: x: -9.973 y: 0.133 z: 0.758 
Accelerometer     [m/s^2]: x: -10.000 y: 0.133 z: 0.777 
Accelerometer     [m/s^2]: x: -9.965 y: 0.145 z: 0.777 
Accelerometer     [m/s^2]: x: -10.000 y: 0.133 z: 0.832 
Accelerometer     [m/s^2]: x: -10.008 y: 0.152 z: 0.785
...
Accelerometer Raw [m/s^2]: x: -10.046 y: 0.182 z: 0.795 
Accelerometer Raw [m/s^2]: x: -10.027 y: 0.153 z: 0.795 
Accelerometer Raw [m/s^2]: x: -9.998 y: 0.153 z: 0.776 
Accelerometer Raw [m/s^2]: x: -9.969 y: 0.172 z: 0.757 
Accelerometer Raw [m/s^2]: x: -9.989 y: 0.144 z: 0.766 
Accelerometer Raw [m/s^2]: x: -9.979 y: 0.105 z: 0.785 
Accelerometer Raw [m/s^2]: x: -9.979 y: 0.153 z: 0.757 
Accelerometer Raw [m/s^2]: x: -9.989 y: 0.144 z: 0.757

diablodale avatar Mar 19 '24 17:03 diablodale