Low FPS depth images (depthplanner)

[nabil-mokhtar]

Question

What's your question?

I'm using airsim for simulating drone to acheive 3d mapping using RTABMAP algorithm , so using kinect_publisher which create ROS node publishes this topics:

• TF
• rgb
• depth
• camera info

my problem is depth topic hz is 0.164 = each frame takes 6 seconds , copmared to rgb topic hz which is 30

Depth FPS

RGB FPS

Include context on what you are trying to achieve

Context details

• Software Specs

  • OS: Ubuntu 20.04
  • AirSim version: v1.7.0
  • Python version: 3.9.7

• Hardware Specs

  • Laptop: Ausu TUF a15
  • CPU: AMD Ryzen 7 4800H with Radeon Graphics
  • GPU: NVIDIA GeForce GTX 1650 Ti
  • Memory 16 GB

---

settings.json

{
"SettingsVersion": 1.2,
"SimMode": "Multirotor",
 "ClockSpeed":1,
"ViewMode":"NoDisplay",
"CameraDefaults": {
  "CaptureSettings": [
    {
      "ImageType": 0,
      "Width": 640,
      "Height": 480,
      "FOV_Degrees": 90,
      "AutoExposureSpeed": 100,
      "AutoExposureBias": 0,
      "AutoExposureMaxBrightness": 0.64,
      "AutoExposureMinBrightness": 0.03,
      "MotionBlurAmount": 0,
      "TargetGamma": 1.0,
      "ProjectionMode": "",
      "OrthoWidth": 5.12
    },
    {
      "ImageType": 1,
      "Width": 640,
      "Height": 480,
      "FOV_Degrees": 90,
      "AutoExposureSpeed": 100,
      "AutoExposureBias": 0,
      "AutoExposureMaxBrightness": 0.64,
      "AutoExposureMinBrightness": 0.03,
      "MotionBlurAmount": 0,
      "TargetGamma": 1.0,
      "ProjectionMode": "",
      "OrthoWidth": 5.12
    }
  ]
  
},


"Vehicles":
{
  "drone":
  {
    "VehicleType": "SimpleFlight",
    "X": 0.0,
    "Y": 0.0,
    "Z": 0.0,
    
    "Sensors":
    {
      "Imu": {
        "SensorType": 2,
        "Enabled": true
      },
      "Gps": {
        "SensorType": 3,
        "Enabled": true
      }
    },
    "Cameras":
    {
      "CAM1":
      {
        "CaptureSettings":
        [
          {
            "ImageType": 0,
            "Width": 640,
            "Height": 480,
            "FOV_Degrees": 87,
            "MotionBlurAmount": 0
          },
          {
            "ImageType": 1,
            "Width": 640,
            "Height": 480,
            "FOV_Degrees": 87,
            "MotionBlurAmount": 0
          }
        ],
        "X": 0.5,
        "Y": 0,
        "Z": -0.15,
        "Roll": 0,
        "Pitch": 0,
        "Yaw": 0
      }
    }
  }
 }
    }

Include details of what you already did to find answers

• After some debugging I found that simGetImages function takes too long duration to get DepthPlanar image with pixels_as_float =true
• after setting "ViewMode":"NoDisplay" in settings.json file the FPS of RGB image improved from 14 FPS to 30 FPS , But the Depth FPS still the same 0.164 FPS

[SabryTarek]

I have the same issue with Depth-Image Depth-Image FPS: 4

[SaundersJE97]

For anyone having issues with this, this is caused by this within 'msgpack' falling back to the python implementation of the packer and unpacker. This is slower than the c implementation.

if os.environ.get('MSGPACK_PUREPYTHON'):
    from msgpack.fallback import Packer, unpackb, Unpacker
else:
    try:
        from msgpack._packer import Packer
        from msgpack._unpacker import unpackb, Unpacker
    except ImportError as e:
        from msgpack.fallback import Packer, unpackb, Unpacker <---- HERE

To solve this, run the following code (clone the repo somewhere safe and don't remove the folder after make):

pip install Cython
git clone -b 0.5.6 https://github.com/msgpack/msgpack-python.git
cd msgpack-python
make