unity-network-hardware-video-decoder icon indicating copy to clipboard operation
unity-network-hardware-video-decoder copied to clipboard

incorrect RGB when streaming depth alligned RGBD (iHD, Ubuntu 20.04)

Open bmegli opened this issue 4 years ago • 6 comments

  • KabyLake
  • only for depth aligned RGB
  • only with iHD
  • both PointCloudRenderer and GPUPointCloudRenderer
# streaming command
./realsense-nhve-depth-color 192.168.0.100 9768 depth 848 480 848 480 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625

image

With i965 everything is ok

LIBVA_DRIVER_NAME=i965 ./realsense-nhve-depth-color 192.168.0.100 9768 depth 848 480 848 480 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625

bmegli avatar Feb 16 '21 18:02 bmegli

From RNHVE encoding pipelines the difference between aligning to color/depth is color format used (the reason)

  • YUYV for color aligned (VAAPI yuyv422)
  • RGBA8 for depth aligned (VAAPI rgb0)

bmegli avatar Feb 16 '21 18:02 bmegli

~~A quick test with HVE encoding raw bgr0/rgb0 (U 20.04, FF 4.2.4, both iHD and i965) suggests that problem is not on encoding side~~ ~~(both raw H.264 play correctly with ffplay)~~

Edit:

  • turns out problem is resolution dependent (manifests at 848x480)

bmegli avatar Feb 17 '21 19:02 bmegli

Possibly related FFmpeg ticket

bmegli avatar Feb 20 '21 17:02 bmegli

Narrowing down the problem (with color streaming only):

  • happens with iHD -> iHD
  • doesn't happen with i965 -> iHD
  • happens at 848x480
  • doesn't happen at 640x360

The fact that it is resolution dependent suggests some kind of stride problem

bmegli avatar Feb 21 '21 18:02 bmegli

Retesting using HVE with specifically 848x480 leads to problem even with FFmpeg playback

  • ok when encoding with i965
  • problem when encoding with iHD

Which means that the problem may be on the encoding side and it is resolution dependent.

bmegli avatar Feb 21 '21 18:02 bmegli

The problem is on encoding side. Nothing to be done here.

bmegli avatar Mar 04 '21 15:03 bmegli