image_proc bug for rectified depth images

[lingfei93]

Hi, I'm using ROS Melodic on Ubuntu 18.04, running a Kinect DK.

I am encountering this very strange bug that I am able to reproduce. On commit 60789f0, when I run slam_RTABMap.launch, the rectified depth cloud that I am able to get is normal. I have attached a picture of the normal looking RTABMap cloud here. As you can see, it is relatively clean.

However, when I use the latest version(commit d761221) of the Azure Kinect ROS Driver, I am getting a very weird rectified depth image.

I am wondering if this is an expected outcome, or which commit caused this error? As there are a lot of commits that happened between the two, I was hoping that someone here could guide me or tell me if this is normal. I have reproduced this issue on two seperate machines running ubuntu 18.04 and ROS melodic.

[skalldri]

That looks like a linear interpolation bug. image_proc/rectify seems to be using interpolation=1 (linear interpolation) when it should be using interpolation=0 (nearest neighbor).

Try changing this chunk in the ROS Launch file:

<!-- Spawn an image_proc/rectify nodelet to rectify the depth image -->
    <node pkg="nodelet" type="nodelet" name="rectify_depth"
          args="load image_proc/rectify manager --no-bond"
          respawn="true">
      <remap from="image_mono"  to="depth/image_raw" />
      <remap from="image_rect"  to="depth/image_rect" />
    </node>

to this:

<!-- Spawn an image_proc/rectify nodelet to rectify the depth image -->
    <node pkg="nodelet" type="nodelet" name="rectify_depth"
          args="load image_proc/rectify manager --no-bond"
          respawn="true">
      <remap from="image_mono"  to="depth/image_raw" />
      <remap from="image_rect"  to="depth/image_rect" />
      <param name="interpolation" value="0" />
    </node>

and see if that makes things better.

[lingfei93]

Yep, added interpolation=0 in the launch file and it seems to work fine! Should I send in a PR for this launch file? Thanks for the help. Just wondering what between the two commits caused this?

Thanks so much for the help!

EDIT: I did some digging, and found that even in the older commit 60789f0, when using interpolation=1(which is the default value), image_proc/rectify was able to do the job. However, the latest commit will only work if interpolation=0 is explictly set!

[jonra1993]

Hello everyone I am using rectify_test.launch and I have same result in pointcloud. Some ghost appear in final pointcloud and there are more when sun light is present. They should be ground points but they appear in the air. I have already interpolation = 0

<!-- Spawn an image_proc/rectify nodelet to rectify the depth image -->
<node if="$(arg depth_rect)"
      pkg="nodelet" type="nodelet" name="rectify_depth"
      args="load image_proc/rectify manager --no-bond"
      respawn="true">
  <remap from="image_mono"  to="depth/image_raw" />
  <remap from="image_rect"  to="depth/image_rect" />
  <param name="interpolation" value="0" />
</node>

[lingfei93]

Hi @jonra1993, your issue seems slightly different from mine. My random dots seem to be concentrated from my camera. I'm not sure about yours? They seem to be more distributed. I have noticed that the Kinect DK does seem to have a few random points past the stated range though.

Good luck, anyhow!