unity-network-hardware-video-decoder
unity-network-hardware-video-decoder copied to clipboard
incorrect RGB when streaming depth alligned RGBD (iHD, Ubuntu 20.04)
- KabyLake
- only for depth aligned RGB
- only with iHD
- both PointCloudRenderer and GPUPointCloudRenderer
# streaming command
./realsense-nhve-depth-color 192.168.0.100 9768 depth 848 480 848 480 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625

With i965 everything is ok
LIBVA_DRIVER_NAME=i965 ./realsense-nhve-depth-color 192.168.0.100 9768 depth 848 480 848 480 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625
From RNHVE encoding pipelines the difference between aligning to color/depth is color format used (the reason)
- YUYV for color aligned (VAAPI yuyv422)
- RGBA8 for depth aligned (VAAPI rgb0)
~~A quick test with HVE encoding raw bgr0/rgb0 (U 20.04, FF 4.2.4, both iHD and i965) suggests that problem is not on encoding side~~ ~~(both raw H.264 play correctly with ffplay)~~
Edit:
- turns out problem is resolution dependent (manifests at 848x480)
Possibly related FFmpeg ticket
Narrowing down the problem (with color streaming only):
- happens with iHD -> iHD
- doesn't happen with i965 -> iHD
- happens at 848x480
- doesn't happen at 640x360
The fact that it is resolution dependent suggests some kind of stride problem
Retesting using HVE with specifically 848x480 leads to problem even with FFmpeg playback
- ok when encoding with i965
- problem when encoding with iHD
Which means that the problem may be on the encoding side and it is resolution dependent.
The problem is on encoding side. Nothing to be done here.