Strange image on RPI 4

[mrischmu]

Hi,

After installing everything, and starting with H264, I get a strange image on VLC:

sudo v4l2rtspserver -H 972 -W 1296 -F 15 -P 8554 /dev/video0

See some outputs: pi@raspberrypi:~/v4l2rtspserver $ v4l2-ctl --list-devices bcm2835-codec-decode (platform:bcm2835-codec): /dev/video10 /dev/video11 /dev/video12

bcm2835-isp (platform:bcm2835-isp): /dev/video13 /dev/video14 /dev/video15 /dev/video16

HD Pro Webcam C920 (usb-0000:01:00.0-1.3): /dev/video0 /dev/video1

pi@raspberrypi:~/v4l2rtspserver $ v4l2-ctl --list-formats ioctl: VIDIOC_ENUM_FMT Type: Video Capture

[0]: 'YUYV' (YUYV 4:2:2)
[1]: 'H264' (H.264, compressed)
[2]: 'MJPG' (Motion-JPEG, compressed)

pi@raspberrypi:~/v4l2rtspserver $ sudo v4l2rtspserver -H 972 -W 1296 -F 15 -P 8554 /dev/video0 2021-07-19 15:46:53,785 [NOTICE] - /home/pi/v4l2rtspserver/main.cpp:294 Version: 0.2.3-26-gd0da079 live555 version:2021.07.10 2021-07-19 15:46:53,786 [NOTICE] - /home/pi/v4l2rtspserver/src/V4l2RTSPServer.cpp:36 Create V4L2 Source.../dev/video0 2021-07-19 15:46:54,022 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:133 driver:uvcvideo capabilities:84a00001 mandatory:4000001 2021-07-19 15:46:54,022 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:136 /dev/video0 support capture 2021-07-19 15:46:54,022 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:139 /dev/video0 support streaming 2021-07-19 15:46:54,023 [WARN ] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:217 /dev/video0: Cannot set size to:1296x972 size is:1280x720 2021-07-19 15:46:54,023 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:225 /dev/video0:H264 size:1280x720 bufferSize:1843200 2021-07-19 15:46:54,024 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:246 fps:1/15 2021-07-19 15:46:54,024 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:247 nbBuffer:0 2021-07-19 15:46:54,024 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:49 Device /dev/video0 2021-07-19 15:46:54,062 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:73 Device /dev/video0 nb buffer:10 2021-07-19 15:46:54,231 [NOTICE] - /home/pi/v4l2rtspserver/src/V4l2RTSPServer.cpp:61 Create Source .../dev/video0 2021-07-19 15:46:54,231 [NOTICE] - /home/pi/v4l2rtspserver/inc/ServerMediaSubsession.h:47 format:video/H264 2021-07-19 15:46:54,231 [NOTICE] - /home/pi/v4l2rtspserver/src/DeviceSource.cpp:93 begin thread 2021-07-19 15:46:54,231 [NOTICE] - /home/pi/v4l2rtspserver/inc/V4l2RTSPServer.h:81 Play this stream using the URL "rtsp://192.168.178.42:8554/unicast"

Thanks for the help!

[mpromonet]

Hi,

This is usually due to packet drop, and this usually drop I-frame first, as it is biggest packet. A possible improvement could be to stream over TCP, this will re-emit the packet that are dropped by the network.

Best Regards, Michel.

[mrischmu]

Hey Michel, Thanks a lot for your quick reply. Additonnal interesting behavior, here is a comparison of the image on qgroundcontrol and VLC at the same time using v4l2rtspserver -W 640 -H 480 -fMJPG F 15 -P 8554 /dev/video0

The image on qgroundcontrol is blue, but has no delay, whereas the image on vlc is clear but has 3seconds of delay

Any idea on what I could do to fix this? Thanks in advance

[mpromonet]

Hi,

Using JPEG instead of H264 avoid the "strange rendering" when loosing frame, it will becomes low frame rate, and use more bandwidth.

VLC use large buffer that introduce delay, you sould try to reduce it, something like network-buffer, network-cache...

For the color I guess either v4l2rtspserver or qgroundcontrol make a wrong interpretation of quantization table, I read several times the rfc2345 but this is not really clear depending if you have 1 or 2 buffer...

Best Regards, Michel.

[e71828]

the same problem

[fujiishigeki]

Hi @mpromonet

I have same problem about Motion JPEG. I use bellow command,

gst-launch-1.0 rtspsrc location=rtsp://192.168.2.2:8554/unicast ! rtpjpegdepay ! jpegparse ! avdec_mjpeg ! videoconvert ! xvimagesink

I get a image with strange color components when using a certain camera. The debug log is bellow, in case of strange color components,

[DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:26 SOF length:17 [INFO] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:42 width:640 height:480 type:0 precision:8 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:49 DQT length:132 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:57 Quantization table idx:0 precision:0 size:64 total size:64 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:77 SOS length:12 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:87 headerSize:589

Next debug log is bellow, in case of collect color components,

[DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:49 DQT length:67 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:57 Quantization table idx:0 precision:0 size:64 total size:128 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:49 DQT length:67 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:57 Quantization table idx:1 precision:0 size:64 total size:128 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:70 DRI restartInterval:10 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:26 SOF length:17 [INFO] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:42 width:640 height:480 type:0 precision:8 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:77 SOS length:12 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:87 headerSize:220

The difference is the number of the DQT and this length (132 or 67) I think that it is the problem of Quantization Table. so I check the DQT on rfc2345, it is described as follows.

When table data is included, the number of tables present depends on the JPEG type field.

I modify the part of handling the DQT for 2 tables and I get collect image. I attached this patch.

Please check this, and can I send a Pull Request? mjpeg_dqt.patch

[mpromonet]

Hi @fujiishigeki

Your modification seems good, Pull Request are always welcomed.

Best Regards, Michel.

[fujiishigeki]

Hi @mpromonet

Thank you for your additional commit. I try previous command with using new HEAD version. The debug log is bellow,

[DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:26 SOF length:17 [INFO] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:42 width:640 height:480 type:0 precision:8 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:49 DQT length:132 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:54 DQT qtable_length:130 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:61 Quantization table idx:0 precision:0 size:64 total size:128 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:54 DQT qtable_length:65 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:61 Quantization table idx:1 precision:0 size:64 total size:128 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:84 SOS length:12 [DEBUG] /opt/github/mpromonet/v4l2rtspserver/src/MJPEGVideoSource.cpp:94 headerSize:589

It is perfect, thank you so much.