v4l2rtspserver
v4l2rtspserver copied to clipboard
mpromonet
v4l2rtspserver breaks on Raspberry Pi OS bullseye
Describe the bug
With the switch to Debian bullseye, Raspberry Pi OS has adopted libcamera which seems to have changed how data is made available via v4l2, with h264, jpeg etc no longer being available.
To Reproduce I installed the current version of Raspberry Pi OS, enabled the camera interface, and captured the following data:
v4l2 output:
$ v4l2-ctl --list-devices
unicam (platform:20801000.csi):
/dev/video0
/dev/video1
/dev/media2
[...]
$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
Size: Discrete 3280x2464
Size: Discrete 1920x1080
Size: Discrete 1640x1232
Size: Discrete 640x480
[1]: 'BG10' (10-bit Bayer BGBG/GRGR)
Size: Discrete 3280x2464
Size: Discrete 1920x1080
Size: Discrete 1640x1232
Size: Discrete 640x480
[2]: 'BA81' (8-bit Bayer BGBG/GRGR)
Size: Discrete 3280x2464
Size: Discrete 1920x1080
Size: Discrete 1640x1232
Size: Discrete 640x480
Example raspistill invocation:
$ libcamera-still -v -o test.jpg
Options:
verbose: 1
info_text:#%frame (%fps fps) exp %exp ag %ag dg %dg
timeout: 5000
width: 0
height: 0
output: test.jpg
post_process_file:
rawfull: 0
preview: default
qt-preview: 0
transform: identity
roi: all
metering: centre
exposure: normal
ev: 0
awb: auto
flush: false
wrap: 0
brightness: 0
contrast: 1
saturation: 1
sharpness: 1
framerate: 30
denoise: auto
viewfinder-width: 0
viewfinder-height: 0
tuning-file: (libcamera)
lores-width: 0
lores-height: 0
encoding: jpg
quality: 93
raw: 0
restart: 0
timelapse: 0
framestart: 0
datetime: 0
timestamp: 0
keypress: 0
signal: 0
thumbnail width: 320
thumbnail height: 240
thumbnail quality: 70
latest:
immediate 0
No connector ID specified. Choosing default from list:
Connector 32 (crtc 0): type 11, 0x0
Connector 49 (crtc 83): type 5, 720x480 (chosen)
Opening camera...
[1:08:43.836864267] [3649] INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
Acquired camera /base/soc/i2c0mux/i2c@1/imx219@10
Configuring viewfinder...
Viewfinder size chosen is 1640x1232
Final viewfinder size is 1640x1232
[1:08:44.136273476] [3649] INFO Camera camera.cpp:937 configuring streams: (0) 1640x1232-YUV420
[1:08:44.139077534] [3651] INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt RG10 Score: 2218 (best 2218)
[1:08:44.140220558] [3651] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt RG10 Score: 2041.48 (best 2041.48)
[1:08:44.141220579] [3651] INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt RG10 Score: 1500 (best 1500)
[1:08:44.142178599] [3651] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt RG10 Score: 5004.81 (best 1500)
[1:08:44.143152619] [3651] INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt pRAA Score: 1718 (best 1500)
[1:08:44.144123639] [3651] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt pRAA Score: 1541.48 (best 1500)
[1:08:44.145088659] [3651] INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt pRAA Score: 1000 (best 1000)
[1:08:44.146052679] [3651] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt pRAA Score: 4504.81 (best 1000)
[1:08:44.147047700] [3651] INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt RGGB Score: 3218 (best 1000)
[1:08:44.148025720] [3651] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt RGGB Score: 3041.48 (best 1000)
[1:08:44.148993740] [3651] INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt RGGB Score: 2500 (best 1000)
[1:08:44.149954760] [3651] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt RGGB Score: 6004.81 (best 1000)
[1:08:44.151476791] [3651] INFO RPI raspberrypi.cpp:620 Sensor: /base/soc/i2c0mux/i2c@1/imx219@10 - Selected mode: 1640x1232-pRAA
Camera streams configured
Buffers allocated and mapped
Viewfinder setup complete
Requests created
[1:08:44.294774761] [3651] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[1:08:44.299829866] [3651] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[1:08:44.301538901] [3651] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
Camera started!
[...]
Similar output and format selection happens when running libcamera-vid -v -o out.h264.
Example v4l2rtspserver invocation:
$ v4l2rtspserver -W 1640 -H 1232 -F 25 -P 8555 -S /dev/video0
log level:500
[NOTICE] /home/pi/v4l2rtspserver/main.cpp:294
Version: 0.2.4-6-g90dfcb1 live555 version:2021.11.10
[NOTICE] /home/pi/v4l2rtspserver/src/V4l2RTSPServer.cpp:37
Create V4L2 Source.../dev/video0
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:133
driver:unicam capabilities:85a00001 mandatory:4000001
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:136
/dev/video0 support capture
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:138
/dev/video0 support read/write
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:139
/dev/video0 support streaming
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:212
/dev/video0: Cannot set pixelformat to:H264 format is:pBAA
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:212
/dev/video0: Cannot set pixelformat to:MJPG format is:pBAA
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:212
/dev/video0: Cannot set pixelformat to:JPEG format is:pBAA
[ERROR] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:83
Cannot init device:/dev/video0
[NOTICE] /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:141
VIDIOC_STREAMOFF: Bad file descriptor
VIDIOC_REQBUFS: Bad file descriptor
Device /dev/video0
[NOTICE] /home/pi/v4l2rtspserver/inc/V4l2RTSPServer.h:163
HLS http://192.168.101.153:8555/ts.m3u8
[NOTICE] /home/pi/v4l2rtspserver/inc/V4l2RTSPServer.h:164
MPEG-DASH http://192.168.101.153:8555/ts.mpd
I'm afraid I don't really know much about v4l2 but would be happy to provide further information if that's at all useful.
Hi @danieltwagner
I don't try libcamera-apps, it seems this is capturing a v4l2 device and compress the stream ?
v4l2rtspserver read a v4l2 device, and support some format (h264, h265, vp8, jpeg, and maybe yuyv). From what we see from v4l2-ctl -d /dev/video0 --list-formats-ext , your device support pBAA, BG10 & BA81. None of these format are supported by v4l2rtspserver.
You may look to v4l2loopback that may be an intermediate device between the compression done by libcamera-apps and v4l2rtspserver.
Best Regards, Michel.
Hello,
right now i have a setup where i use rpos from Breeze. Unfortunatly i run into the same problem as @danieltwagner. My v4l2rtspServer is working well on Buster, the second i switch to Bullseye with the new way libcamera is implemented my setup is in trouble. My attempts to fix this issue ended up in failure. If someone is capable of fixing this i would appreciate it. The moment i get a working workaround i will post it here.
Greetings, Jasp
Hello,
My workaround now is mentioned here: https://forums.raspberrypi.com/viewtopic.php?t=323390&sid=9327da4837fe5a986060d9e3c64c46c1
It is just some simple config setup, which will end up in rebooting the old v4l2driver.
The Solution was to edit /boot/config.txt, remove the line "camera_auto_detect=1", and add "start_x=1" and "gpu_mem=128".
I hope that this might help someones setup.
Greetings, Jasp
Hello,
My workaround now is mentioned here: https://forums.raspberrypi.com/viewtopic.php?t=323390&sid=9327da4837fe5a986060d9e3c64c46c1
It is just some simple config setup, which will end up in rebooting the old v4l2driver.
The Solution was to edit /boot/config.txt, remove the line "camera_auto_detect=1", and add "start_x=1" and "gpu_mem=128".
I hope that this might help someones setup.
Greetings, Jasp
Good job finding the temporary fix, but as you pointed out in your original post, it rolls back to the old camera driver, so it's really a "workaround". I have the same concern and I know any new drivers will break my camera streaming setup, so I'm holding out until everything is ready.
You may look to v4l2loopback that may be an intermediate device between the compression done by libcamera-apps and v4l2rtspserver.
I have attempted this configuration on Raspberry PI B3+ and it worked somewhat but did not produce the video upon the connection to RTSP server.
Issue is resolved when editing:
sudo nano /boot/config.txt
#camera_auto_detect=1
start_x=1
gpu_mem=128
Reboot and the driver is now not unicam but familiar:
mmal service 16.1 (platform:bcm2835-v4l2-0):
/dev/video0
When camera_auto_detect=1 is enabled the output looks like this:
v412-ctl --list-devices
looks like this:
v4l2-ctl --list-devices
unicam (platform:3f801000.csi):
/dev/video0
/dev/video1
/dev/media2
bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12
/dev/video18
/dev/media1
bcm2835-isp (platform:bcm2835-isp):
/dev/video13
/dev/video14
/dev/video15
/dev/video16
/dev/media0
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video2
If anyone wants to reproduce:
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install cmake liblog4cpp5-dev libv4l-dev libssl-dev git raspberrypi-kernel-headers autoconf libtool -y
git clone https://github.com/umlaeute/v4l2loopback.git; (cd v4l2loopback/ ; make clean; sudo make install; sudo depmod -a ; sudo modprobe v4l2loopback)
git clone https://github.com/mpromonet/v4l2rtspserver.git; (cd v4l2rtspserver/ ; cmake . ; make ; sudo make install)
git clone https://github.com/mpromonet/v4l2tools.git ; (cd v4l2tools/ ; make ; make install)
sudo modprobe v4l2loopback
# Find the device that v4l2loopback is running on with helper command below or manually `v4l2-ctl --list-devices` looking for v4l2loopback.
LOOPBACK_DEVICE=$(v4l2-ctl --list-devices | grep v4l2loopback -A1 | awk 'FNR == 2 {print $1}')
./v4l2tools/v4l2compress -fH264 /dev/video0 $LOOPBACK_DEVICE &
v4l2rtspserver -W 640 -H 480 -F 15 -P 8554 $LOOPBACK_DEVICE
I tried loading driver... With no success
which in docs camera seems to be the one for this
$ sudo modprobe bcm2835-unicam
$ sudo modprobe v4l2loopback
$ v4l2compress -fH264 /dev/video0 /dev/video17 & v4l2rtspserver /dev/video17 &
With no success I connected that one on https://github.com/umlaeute/v4l2loopback but even so it is not working...
Raspberry pi 4 8gb bullseye lite (libcamera)
Devices
~ » v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12
/dev/video18
/dev/media2
bcm2835-isp (platform:bcm2835-isp):
/dev/video13
/dev/video14
/dev/video15
/dev/video16
/dev/media1
unicam (platform:fe801000.csi):
/dev/video0
/dev/video1
/dev/media0
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video2
looking forward to making it work on bullseye. My output
~ » v4l2compress -fH264 /dev/video0 /dev/video2 & v4l2rtspserver /dev/video2 &
[1] 29119
[2] 29120
2021-11-21 16:53:46,462 [NOTICE] - src/V4l2Device.cpp:133
driver:unicam capabilities:85a00001 mandatory:4000001
2021-11-21 16:53:46,462 [NOTICE] - src/V4l2Device.cpp:136
/dev/video0 support capture
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2Device.cpp:138
/dev/video0 support read/write
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2Device.cpp:139
/dev/video0 support streaming
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2Device.cpp:225
/dev/video0:pBAA size:1640x1232 bufferSize:2562560
2021-11-21 16:53:46,463 [NOTICE] - src/V4l2MmapDevice.cpp:49
Device /dev/video0
--------------------------------------------------------------------------------------------------------------------------------------------
~ » 2021-11-21 16:53:46,490 [NOTICE] - src/V4l2MmapDevice.cpp:73
Device /dev/video0 nb buffer:10
log level:500
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/main.cpp:294
Version: 0.2.4-7-g9638a15-dirty live555 version:2020.01.19
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/src/V4l2RTSPServer.cpp:37
Create V4L2 Source.../dev/video2
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:133
driver:v4l2 loopback capabilities:85208003 mandatory:4000001
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:135
/dev/video2 support output
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:136
/dev/video2 support capture
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:138
/dev/video2 support read/write
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:139
/dev/video2 support streaming
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:192
/dev/video2: Cannot get format Invalid argument
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:192
/dev/video2: Cannot get format Invalid argument
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:192
/dev/video2: Cannot get format Invalid argument
[ERROR] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2Device.cpp:83
Cannot init device:/dev/video2
[NOTICE] /home/ro/Documents/code/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:141
VIDIOC_STREAMOFF: Bad file descriptor
VIDIOC_REQBUFS: Bad file descriptor
Device /dev/video2
[2] + 29120 done v4l2rtspserver /dev/video2
--------------------------------------------------------------------------------------------------------------------------------------------
~ » 2021-11-21 16:53:46,544 [NOTICE] - src/V4l2Device.cpp:133
driver:v4l2 loopback capabilities:85208003 mandatory:4000002
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:135
/dev/video2 support output
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:136
/dev/video2 support capture
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:138
/dev/video2 support read/write
2021-11-21 16:53:46,545 [NOTICE] - src/V4l2Device.cpp:139
/dev/video2 support streaming
2021-11-21 16:53:46,551 [NOTICE] - src/V4l2Device.cpp:225
/dev/video2:H264 size:1640x1232 bufferSize:8085504
2021-11-21 16:53:46,551 [NOTICE] - src/V4l2MmapDevice.cpp:49
Device /dev/video2
2021-11-21 16:53:46,551 [NOTICE] - src/V4l2MmapDevice.cpp:73
Device /dev/video2 nb buffer:2
2021-11-21 16:53:46,560 [NOTICE] - src/v4l2compress.cpp:74
Start Compressing to /dev/video2
[1] + 29119 segmentation fault v4l2compress -fH264 /dev/video0 /dev/video2
