[HOW-TO] Customize the FPS of a recorded video using H264Encoder

[azsde]

Hello,

I am trying to record a video at the highest framerate supported by my camera V3 which is supposed to be capable of doing 1080p@50FPS according to the documentation.

But the FPS I'm trying to apply doesn't seem to work and I always get an output file that is 30FPS:

Here is my code:

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder, Quality

from libcamera import controls


picam2 = Picamera2()
picam2.video_configuration.controls.FrameRate = 50.0
video_config = picam2.create_video_configuration(main={"size": (1920, 1080)})
picam2.configure(video_config)
picam2.set_controls({"AfMode": controls.AfModeEnum.Continuous})
h264encoder = H264Encoder()
picam2.start_recording(h264encoder, "test.h264", quality=Quality.VERY_HIGH)
sleep(10)
picam2.stop_recording()

What am I doing wrong ?

I thank you in advance for your help.

Regards,

Azsde.

[davidplowman]

Hi, please try

video_config = picam2.create_video_configuration(main={"size": (1920, 1080)}, controls={'FrameRate': 50})

(and you probably don't need the line above.) Does that make a difference?

[azsde]

It doesn't seem to affect the reported framerate in MediaInfo (which is still at 30.000 FPS).

Moreover, when using ffmpeg to convert it into a mp4:

ffmpeg -i input.h264 -r 50 -c:v copy output.mp4

The duration is not correct : I filmed a timer of 10 seconds, and the converted video is only 7 seconds longs.

[davidplowman]

There are always difficulties like this with unencapsulated ("raw") h.264 streams, because they don't have "real" timestamps in them. Does it help if you create the h.264 encoder with

h264encoder = H264Encoder(framerate=50, enable_sps_framerate=True)

?

[azsde]

Ok, so now the .h264 file correctly is reporting 50 FPS:

But either when using MP4Box or ffmpeg to make it into a playable .mp4 file, the duration is not correct and the footage is sped up (it should be around 10s and is recognized to be 5 seconds long by VLC).

Am I missing something here ?

Here is an archive containing the produced h264 file: test-10s.zip

I've tried the following commands:

• MP4Box -add test-10s.h264 test-10s-boxmp4.mp4
• ffmpeg -i test-10s.h264 -r 50 -c:v copy test-10s-ffmpeg.mp4

[davidplowman]

Maybe try saving a timestamp file alongside the h.264 file so that we can check the video really is running at 50fps? (Add pts="timestamps.txt" to the start_recording() call.)

[azsde]

Here's the zip file of the produced video alongside with the timestamp: Test with timestamp.zip

I don't really know what to look for in the timestamps.txt file.

Also, it seems that running the following command (from the official documentation https://www.raspberrypi.com/documentation/computers/camera_software.html#high-framerate-capture) produces a .h264 file that can be "converted" into a mp4 file that also has an incorrect duration if "converted" with ffmpeg or MP4Box.

libcamera-vid --level 4.2 --framerate 60 --width 1920 --height 1080--save-pts timestamp.pts -o video.264 -t 10000 --denoise cdn_off -n

That being said, using mkvmerge to produce an mp4 file does seem to work when using the libcamera-vid binary.

The timestamps produced by picamera2 are missing a header to be usable by mkvmerge:

azsde@DESKTOP-R721E5F:/mnt/c/Users/azsde/Downloads$ mkvmerge -o test.mkv --timecodes 0:timestamps.txt test.h264
mkvmerge v65.0.0 ('Too Much') 64-bit
'test.h264': Using the demultiplexer for the format 'AVC/H.264'.
Error: The timestamp file 'timestamps.txt' contains an unsupported/unrecognized format line. The very first line must look like '# timestamp format v1'.

Manually adding:

# timestamp format v2

At the top of the timestamp file produced by picamera2 seems to solve the issue and mkvmerge can produce a playable .mp4 that has the correct duration.

[davidplowman]

Looking at the timestamp file (which contains frame timestamps in milliseconds), the camera does seem to be delivering frames every 20ms (actually very marginally less). But there are lots of jumps when a whole second or so of frames go missing. So it seems to be that the problem is that there are large numbers of frame drops. What kind of Pi is this?

I would expect the correct duration if you give the timestamp file to mkverge, only there will presumably be numerous small pauses where the missing frames should be.

[azsde]

This a Raspberry Pi 3b+, is it too weak to handle such a video stream ?

[davidplowman]

I'm afraid I'm starting to get a bit confused about what you've run using Python, and what you've run using libcamera-vid.

Are you saying that this libcamera-vid command works, and you see no frame drops at 50fps?

libcamera-vid --level 4.2 --framerate 60 --width 1920 --height 1080--save-pts timestamp.pts -o video.264 -t 10000 --denoise cdn_off -n

[azsde]

Sorry if I'm not clear enough, let me try again:

This command:

libcamera-vid --level 4.2 --framerate 60 --width 1920 --height 1080 --save-pts timestamp.pts -o video.264 -t 10000 --denoise cdn_off -n

Produces a 60 FPS h264 file (not 50 FPS).

Using ffmpeg / mp4box to get a playable video file result in a MP4 file that do not have the expected duration (around 6 seconds instead of 10 seconds):

MP4Box -add video.264 video-boxmp4.mp4
ffmpeg -i video.264 -r 50 -c:v copy video-ffmpeg.mp4

Using mkvmerge to get a playable video result in a MKV file that does have the expected duration, BUT has the wrong FPS:

mkvmerge -o video-mkvmerge.mkv --timecodes 0:timestamp.pts video.264

You are probably right about the dropped frames, it seems that depending on the way of turning the h264 file into a playable video file, it will either stay true to the FPS and affect the duration, or will change the FPS in order to preserve the total duration.

I do have access to a pi4, should I try the same code on it ?

[tvoverbeek]

Isn't it just a memory issue? 3B+ has 'only' 1GB. Pi4 probably has more (2, 4, or 8GB). Try the Pi4. My guess is it will behave better.

[azsde]

I just tried with a Pi 4 8Gb, I still have issues to "convert" the .h264 file to a .mp4 file using ffmpeg / mp4box, the output videos are indeed 60FPS but they play too fast.

When using mkvmerge, the playback speed is correct, however the output file is not 60 FPS but rather 40ish FPS.

Here's the complete code I'm using:

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder, Quality

from time import sleep
from libcamera import controls

picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (1920, 1080)}, controls={'FrameRate': 50})
picam2.configure(video_config)
picam2.set_controls({"AfMode": controls.AfModeEnum.Continuous})
h264encoder = H264Encoder(framerate=50, enable_sps_framerate=True)
picam2.start_recording(h264encoder, "test-50.h264", quality=Quality.VERY_HIGH, pts="timestamps-50.txt")
sleep(60)
picam2.stop_recording()

And here are the commands used to turn the h264 into a .mp4 or a .mkv:

# Produces a 50FPS video with a duration of 50 seconds
ffmpeg -i test-50.h264 -r 50 -c:v copy test-ffmpeg-50.mp4

# Produces a 50FPS video with a duration of 50 seconds
MP4Box -add test-50.h264 test-mp4box-50.mp4

# Produces a 42.19 FPS video with a duration of 59 seconds
mkvmerge -o test-mkv-50.mkv --timecodes 0:timestamps-50.txt test-50.h264 

[davidplowman]

One thing you have to do at high framerates is turn off the extra denoise processing, which is relatively slow (the libcamera-vid command you were using does this). In Python, add 'NoiseReductionMode': 1 to the controls when you create the configuration.

I would say the first thing to check is that libcamera-vid will run and give you the correct framerate reliably with no drops, judged by checking the timestamp file. Also be sure to run without a preview window. Then let's move on to Python. Maybe use format "YUV420", and perhaps start with lower resolutions/bitrates to begin with. Anyway, so I think your configuration would be something like this:

video_config = picam2.create_video_configuration(main={"size": (1920, 1080), 'format': 'YUV420'},
                                                 controls={'FrameRate': 50, 'NoiseReductionMode': 1})

[azsde]

Hello,

I had time to work on the subject again,the issue doesn't seem to come from picamera as libcamera-vid produces the same result:

libcamera-vid --level 4.2 --framerate 50 --width 1920 --height 1080 --save-pts timestamp-libcam.pts -o video-libcam.264 -t 60000 --denoise cdn_off -n

Using MediaInfo on the "raw" h264 file shows that the produced file has the expected FPS.

However, when trying to play the h264 file using VLC, the output video doesn't have any duration and play faster than it should (like if the video was sped up).

"Converting" the h264 file using ffmpeg also result in a 50FPS video, but with an incorrect duration and it play faster that it should as well.

"Converting" the h264 file using mvkmerge result in a 32ish FPS video with an correct duration and it plays at a normal rate.

Anyway, I'll try to get support on the Raspberry Pi forums as it is not an issue related to picamera2.

[kodonnell]

I'd try recording at a much lower resolution and/or quality. If you get the FPS you're expecting with that, then it's likely a performance issue, which seems quite possible. If not, then you can dig deeper.

Generating a timestamps.txt is about the only reliable way I've found to validate the framerate. Each represents the timestamp for each frame, so you can figure out the FPS. (As in your example above, they're 20ms apart, apart from the frame drops, which timestamps.txt is also useful for detecting.)

[davidplowman]

Remember that ".h264" is not a proper file format. It contains video frames but without timestamps, so a player can choose to play it at any rate that it wants. Also note that for the last few years, vlc has been unable to play streams without timestamps properly. It skips stuff, produces garbled output, all kinds of nonsense happens. If you want to play ".h264" files, I recommend a player other than vlc (e.g. ffplay).

[PythoErgo]

Hello there, I'm having the exact same problem as @azsde, here's my simple testing code:

from picamera2 import Picamera2, encoders
from picamera2.encoders import H264Encoder
import signal
import sys

picam2 = Picamera2()

# Define 480p at 48fps
cfg = {"size": (640, 480), "fps": 48}
filename = "480p48_test.h264"

# Timeout handler
def timeout_handler(signum, frame):
    print("\n[INFO] Timeout reached. Stopping recording...")
    picam2.stop_recording()
    print("[INFO] Recording complete. Saved as 480p48_test.h264")
    sys.exit(0)

# Set the timeout signal
signal.signal(signal.SIGALRM, timeout_handler)
signal.alarm(10)  # 10-second timeout

# Create and apply configuration
video_config = picam2.create_video_configuration(
    main={"size": cfg["size"], "format": "YUV420"}
)
picam2.configure(video_config)
picam2.set_controls({"FrameRate": cfg["fps"]})

# Encoder with SPS frame rate signaling
h264encoder = H264Encoder(framerate=cfg["fps"], enable_sps_framerate=True)

# Begin recording
print("[INFO] Recording for 10 seconds at 480p 48fps...")
picam2.start()
picam2.start_recording(h264encoder, filename)

# Wait indefinitely; timeout handler will trigger exit
signal.pause()

Very basic, common code here. When trying to record for 10 seconds I checked the actual duration using ffmpeg, which says the video is 343 frames and 7.12 seconds long (343/7.12=48.17fps, correct framerate but wrong duration). I've tried two other similar methods like picam2.video_configuration.controls.FrameRate = 48.0 and picam2.set_controls({"FrameRate": 48}), both of which do not actually set the fps to 48 but instead to the default 30fps.

A quick workaround to this would be to use subProcess and simply run the console command (libcamera-vid) with parameters from input(), but I'm trying to make one big Python script for some smart glasses (needs code for display, Bluetooth, video recording) I am making, and I'm not sure if that would work.

[davidplowman]

@PythoErgo Thanks for the question. I would certainly agree that running rpicam-vid in a sub-process sounds much clunkier and gives you less control, so I would definitely try and avoid that.

I might start by confirming a couple of basic things:

• What framerate are you really getting? Maybe ignore the recording for now and try this:

# ...
picam2.start()
picam2.capture_metadata()  # don't time very first frame, it can be a bit slow

count = 0
start_time = time.monotonic()
while time.monotonic() - start_time < 10:
    picam2.capture_metadata()
    count += 1

print(count / 10, "fps")

• Check how many frames the encoder is seeing. When the recording finishes, I think you can look at encoder.frames_encoded.

You might also want to set the framerate as part of the configuration, like so:

video_config = picam2.create_video_configuration(
    main={"size": cfg["size"], "format": "YUV420"},
    controls={'FrameRate': cfg["fps"]}
)

as this will take effect as soon as the camera starts (setting it later will cause it to wait a few frames before it changes).

[PythoErgo]

@davidplowman Thanks for replying so quickly! Let's go over some results and answers then.

1. With the code below, I am apparently getting roughly 44 fps. I know the camera can do 48fps at 640x480 resolution because I've ran the libcamera-vid command several times before in the console, all with a solid 48 fps.

from picamera2 import Picamera2, encoders
from picamera2.encoders import H264Encoder
import sys
import time

picam2 = Picamera2()

# Define 480p at 48fps
cfg = {"size": (640, 480), "fps": 48}
filename = "480p48_test.h264"

# Create and apply configuration
video_config = picam2.create_video_configuration(
    main={"size": cfg["size"], "format": "YUV420"}
)
picam2.configure(video_config)
picam2.set_controls({"FrameRate": cfg["fps"]})

# Encoder with SPS frame rate signaling
h264encoder = H264Encoder(framerate=cfg["fps"], enable_sps_framerate=True)

# Begin recording
print("[INFO] Recording for 10 seconds at 480p 48fps...")
picam2.start()
picam2.capture_metadata()  # don't time very first frame, it can be a bit slow

count = 0
start_time = time.monotonic()
while time.monotonic() - start_time < 10:
    picam2.capture_metadata()
    count += 1

print(count / 10, "fps")

2. With the code below (not sure if I possibly missed something in your suggestion, partly pieced together with ChatGPT), it recorded 355 frames for 11.79 seconds, seems like the timing is off, and 355/11.79 is still 30 fps.

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
import time

picam2 = Picamera2()

# Define 480p at 48fps
cfg = {"size": (640, 480), "fps": 48}
filename = "480p48_test.h264"

# Timeout handler
def timeout_handler(signum, frame):
    print("\n[INFO] Timeout reached. Stopping recording...")
    picam2.stop_recording()
    print("[INFO] Recording complete. Saved as 480p48_test.h264")
    sys.exit(0)

# Create and apply configuration
video_config = picam2.create_video_configuration(
    main={"size": cfg["size"], "format": "YUV420"},
    controls={'FrameRate': cfg["fps"]}
)
picam2.configure(video_config)

# Begin recording
print("[INFO] Recording for 10 seconds at 480p 48fps...")
picam2.start()

# Prime the first frame (optional but recommended for timing accuracy)
picam2.capture_metadata()

encoder = H264Encoder()
picam2.start_recording(encoder, filename)

# Time-accurate 10-second recording loop
start_time = time.monotonic()
while time.monotonic() - start_time < 10:
    time.sleep(0.01)  # slight sleep to avoid busy-waiting

picam2.stop_recording()
print("[INFO] Recording complete. File saved as:", filename)

Note: edited to fix the code tags. Pytho note: How do I do these code tags...

[davidplowman]

Hi again, you don't say what kind of a Pi and what kind of a camera you have. Pasting the output of rpicam-hello --list-cameras would be helpful, just to see what camera modes you have available.

If you have a non-Pi 5, then it may well be worth adding "NoiseReductionMode": 3 to the controls that you set at the start. Really, you want that reporting the expected framerate before you do anything else. Might be worth printing out picam2.camera_configuration() once you've configured the camera as well, just to check what mode it selected.

Under some circumstances, increasing the number of buffers in the camera queue can help (add buffer_count=16 when you create the configuration).

In the final example, I suspect that suspending/waking the main Python thread 100 times per second may not be helping, but in any case, let's make the other example work first.

(PS. for a block of code, use a line with just three consecutive backquotes to start/end it)

[PythoErgo]

I am using a Raspberry Pi Zero W. After running rpicam-hello --list-cameras I got this: As for your other suggestion I simply added print(picam2.camera_configuration()) before starting the camera (not sure if this is wrong or not) and got this:

[davidplowman]

Thanks for that. The camera mode selection and everything looks fine. So I'd make that noise reduction and buffer_count change that I suggested, and then see what fps you get running the first test case (so no recording yet).

[PythoErgo]

So something like this?

video_config = picam2.create_video_configuration(
    main={"size": cfg["size"], "format": "YUV420"},
    buffer_count=16
)
picam2.configure(video_config)
picam2.set_controls({"FrameRate": cfg["fps"], "NoiseReductionMode": 3})

35.3 fps... I don't think that's right, honestly. But there you go.

[davidplowman]

Hmm, that doesn't seem great. Maybe I need to try and find a Zero W to try it. I tend not to have them lying around because they are pretty slow at running Python code, but I'll have a look round.

Just to check - you're running Raspberry Pi OS Lite (no desktop), Bookworm 32-bit, right?

[PythoErgo]

I am running Raspberry Pi OS Desktop 32bit, but I'm on console boot right now and CPU, memory, etc levels are healthy. With desktop boot I'd have zero chance of video recording, haha.

[davidplowman]

Well, I've had a go now, and I can't get it to go faster than about 43fps, and that's when it's not encoding. There are certainly some slightly strange behaviours, like increasing the buffer count making it worse, and I don't really understand that. The Pi Zero is a very constrained platform, in terms of CPU and memory bandwidth, but I don't really see how that could be causing memory pressure (or something??) somewhere else.

So I don't really think I have an answer other than that a Pi Zero seems to be underpowered for this. I don't suppose you have a Pi Zero 2? That's a much more capable platform.

[PythoErgo]

Alright, thank you so much for the help! I will try something like 35 fps and get back to you alright? As for the Pi Zero 2 W, seems like it's time to upgrade!

[PythoErgo]

Well, I just got the Raspberry Pi Zero 2 W. Question still remains, how to control FPS in Python? If the mystery is still unsolved then I will have to simply work with what I'm given.

[davidplowman]

I ran the following on my Pi Zero 2 and v1 camera

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import Output
import time

picam2 = Picamera2()
config = picam2.create_video_configuration(
    {'size': (640, 480), 'format': 'YUV420'},
    controls={'FrameRate': 60, 'NoiseReductionMode': 3}
)
picam2.configure(config)
encoder = H264Encoder()
output = Output()

picam2.start_recording(encoder, output)
time.sleep(10)
picam2.stop_recording()

print("Frames encoded:", encoder.frames_encoded)

and it reports Frames encoded: 614 when it finishes.

[PythoErgo]

Just to confirm, you're using this camera module right?

I got 319 frames with your exact code. I'm using the Pi Zero W though, want me to try switching to the Pi Zero 2 W?

Edit: second test got 343, third got 342 frames.