raspberrypi
[HOW-TO] Capture still image while recording in YUV420 format
Hello,
I am trying to capture an image while recording, but I have issues doing so because I am recording in YUV420 format.
import time
from libcamera import Transform
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder, Quality
picam2 = Picamera2()
cfg = picam2.create_video_configuration(main={"size": (1920, 1080), "format": "YUV420"},
controls={
'NoiseReductionMode': 1},
transform=Transform(hflip=1, vflip=1))
picam2.configure(cfg)
h264encoder = H264Encoder()
picam2.start()
picam2.start_recording(h264encoder, "test.mp4", quality=Quality.HIGH)
time.sleep(5) # Record for 5 seconds
request = picam2.capture_request()
request.save("main", "test.jpg")
request.release()
picam2.stop_recording()
Traceback (most recent call last):
File "/home/azsde/RCFI/test.py", line 19, in <module>
request.save("main", "test.jpg")
File "/usr/lib/python3/dist-packages/picamera2/request.py", line 173, in save
return self.picam2.helpers.save(self.make_image(name), self.get_metadata(), file_output,
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/request.py", line 165, in make_image
return self.picam2.helpers.make_image(self.make_buffer(name), self.config[name], width, height)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/request.py", line 244, in make_image
raise RuntimeError(f"Stream format {fmt} not supported for PIL images")
RuntimeError: Stream format YUV420 not supported for PIL images
How can I resolve this ?
I thank you in advance for your help !
Hi, this works for me on Picamera2 version 0.3.27 - and in fact I remember doing quite a lot of work to make direct saving of YUV420 possible. Can you double-check what version you have (apt list python3-picamera2). Thanks.
Yeah I'm was on an older version (0.3.23-1), after upgrading it works correctly :)
Silly question, does capture_request takes a still of the video or is it a proper "photo capture" ?
It's a regular photo capture, just as if the video isn't being recorded.
Awesome! I didn't know it was possible to both have a continuous recording and a regular picture taken !
Edit: I may have misunderstood how things work, if I want the picture with the best quality (resolution ?) possible, I still need to call switch_mode_and_capture_image right ?
Because this doesn't work:
import time
from libcamera import Transform
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder, Quality
picam2 = Picamera2()
cfg = picam2.create_video_configuration(main={"size": (1920, 1080), "format": "YUV420"},
controls={
'NoiseReductionMode': 1},
transform=Transform(hflip=1, vflip=1))
picam2.configure(cfg)
h264encoder = H264Encoder()
still_capture_config = picam2.create_still_configuration()
picam2.start()
picam2.start_recording(h264encoder, "test.mp4", quality=Quality.HIGH)
time.sleep(5) # Record for 5 seconds
picam2.switch_mode_and_capture_image(still_capture_config, "test.jpg")
picam2.stop_recording()
Traceback (most recent call last):
File "/home/azsde/RCFI/test.py", line 20, in <module>
picam2.switch_mode_and_capture_image(still_capture_config, "test.jpg")
File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 2280, in switch_mode_and_capture_image
return self.dispatch_functions(functions, wait, signal_function, immediate=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1399, in dispatch_functions
return job.get_result(timeout=timeout) if wait else job
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/job.py", line 87, in get_result
return self._future.result(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/usr/lib/python3/dist-packages/picamera2/job.py", line 56, in execute
done, result = self._functions[0]()
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 2271, in capture_image_and_switch_back_
done, result = self.capture_image_(name)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 2196, in capture_image_
result = request.make_image(name)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/picamera2/request.py", line 182, in make_image
Exception during process_requests()
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 85, in handle_request
picam2.process_requests(self)
File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1338, in process_requests
encoder.encode(encoder.name, req)
File "/usr/lib/python3/dist-packages/picamera2/encoders/encoder.py", line 260, in encode
self._encode(stream, request)
File "/usr/lib/python3/dist-packages/picamera2/encoders/libav_h264_encoder.py", line 151, in _encode
frame = av.VideoFrame.from_numpy_buffer(m.array, format=self._av_input_format, width=self.width)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/video/frame.pyx", line 408, in av.video.frame.VideoFrame.from_numpy_buffer
File "av/utils.pyx", line 76, in av.utils.check_ndarray
ValueError: Expected numpy array with ndim `2` but got `3`
raise RuntimeError(f'Stream {name!r} is not defined')
RuntimeError: Stream 'test.jpg' is not defined
Exception in thread Thread-2 (thread_func):
Traceback (most recent call last):
File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
self.run()
File "/usr/lib/python3.11/threading.py", line 975, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 29, in thread_func
callback(picam2)
File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 85, in handle_request
picam2.process_requests(self)
File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1338, in process_requests
encoder.encode(encoder.name, req)
File "/usr/lib/python3/dist-packages/picamera2/encoders/encoder.py", line 260, in encode
self._encode(stream, request)
File "/usr/lib/python3/dist-packages/picamera2/encoders/libav_h264_encoder.py", line 151, in _encode
frame = av.VideoFrame.from_numpy_buffer(m.array, format=self._av_input_format, width=self.width)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/video/frame.pyx", line 408, in av.video.frame.VideoFrame.from_numpy_buffer
File "av/utils.pyx", line 76, in av.utils.check_ndarray
ValueError: Expected numpy array with ndim `2` but got `3`
Right, so this is where we have to deal with camera modes and the limitations caused by having to choose just one of them.
While you're recording a video, you can't switch camera modes. Even if you could "suspend" the video while switching mode for the still capture, and "resume" afterwards, you have a giant gap of a second of more in your video file.
So unfortunately you have to choose:
- A camera mode with a nice fast framerate for your video. But these are typically 2x2 binned modes, so will come with reduced resolution.
- Or the full resolution camera mode for your still images, but these will have a much lower framerate, depending on the sensor.
One day in the (currently still distant) future we may have 4-lane camera boards and camera drivers, at which point we'll be able to receive full resolution frames at much faster framerates, which will solve the problem, but until then I can't really think of any alternatives.
I see, thank you very much for your answer.
So the solution if I want max resolution picture would be to pause/stop the recording, take the picture, and resume/start a new recording ?
Yes. As things stand, I think you'd have to stop the recording, and start another afterwards.
I suppose one could have a feature where the recording is "paused", and "resumed" after - then you'd end up with just a single file, rather than two. You'd have to think how to handle the gap in the recording - would you actually leave that in the file, or would you "snip it out" as you go?
Yes. As things stand, I think you'd have to stop the recording, and start another afterwards.
I suppose one could have a feature where the recording is "paused", and "resumed" after - then you'd end up with just a single file, rather than two. You'd have to think how to handle the gap in the recording - would you actually leave that in the file, or would you "snip it out" as you go?
I'd be fine with a gap in a single video file, but this would require stitching the videos together after the second recording stops I guess ?
As things stand, yes, you'd have to stitch files together. If it's an unencapsulated H.264 stream, that that's probably just a case of appending the files together. For an mp4 file, I suppose it will require ffmpeg - though at least you don't need to re-encode, so it should be quick.
As a hack, you could try setting encoder._running = False to pause the recording before you switch mode, and then encoder._running = True once you're back in the video mode. It might work, or it might not. If it worked, we could think about formalising it.
@davidplowman I've tried with the following code:
import time
from libcamera import Transform
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder, Quality
picam2 = Picamera2()
cfg = picam2.create_video_configuration(main={"size": (1920, 1080), "format": "YUV420"},
controls={
'NoiseReductionMode': 1},
transform=Transform(hflip=1, vflip=1))
picam2.configure(cfg)
h264encoder = H264Encoder()
still_capture_config = picam2.create_still_configuration()
picam2.start()
print("Start recording for 5 seconds ...")
picam2.start_recording(h264encoder, "test.mp4", quality=Quality.HIGH)
time.sleep(5) # Record for 5 seconds
print("Pause to take a still image ...")
h264encoder._running = False
image = picam2.switch_mode_and_capture_image(still_capture_config)
image.save("test.jpg")
print("Done, resuming video recording ...")
h264encoder._running = True
time.sleep(5)
print("Video recording ended")
picam2.stop_recording()
Unfortunately, this produces an unplayable video.
Hi, a couple of things here.
I think calling the output file "test.mp4" isn't enough to make it an mp4 file. Under the hood, it'll be opening a FileOutput and so you'll actually end up with an unencapsulated H.264 stream written to a flat file, that just happens to have a name ending in ".mp4".
For what it's worth, I think the file is "OK" as an H.264 stream, but it's obviously not a valid mp4 file.
The better way to do this would be using a PyavOutput. Replace the start_recording line by
output = PyavOutput("test.mp4")
picam2.start_recording(h264encoder, output, quality=Quality.HIGH)
This will give you a genuine mp4 file, and, so far as I can tell, this now works.
One very minor tip: maybe resume the encoder before saving image, so as to marginally reduce the gap in the recording.