GoSecure
fix(359): Support fractional fps.
This pull request fixes #359
I've done two things:
- Default FPS is now 25. (PS: It looks like a recent refactor makes it impossible to configure the FPS for the MP4 converter)
- Use fractional division and keep track of the drift. Whenever we drift by more than one frame, an extra frame is added.
I'm not sure if the drift was a real issue though since we only updated the previous timestamp whenever at least one frame was rendered. In other words, if the timestamp between 2 PDUs is smaller than the frame rate delta, we would just let the delta increase until we've gone past the next integer frame, then render the screen at the current PDU.
The only thing that might happen is that at low frame rates, some information is lost because the screen changes faster than our frame rate can observe. RDP doesn't really have a concept of framerate, updates are sent as they happen, so it's difficult to map it to a constant frame rate.
The code looks fine. I'll run it on a large conversion job and check drift values to confirm we are fixing the right behavior I've seen.
Based on early testing, I think this needs to be revisited. I already spoke to the submitter about this privately.
Highlight: Video duration is 10 seconds more for a 34 seconds video (if we trust the master's number). This is 33%+ longer. Does it make sense?
Below is some evidence collected on a small capture that I can share privately.
Little performance impact, most likely attributable to duration.
Original:
(venv) olivier@barachois:~/gosecure/research/projets/2021-10_pyrdp-video-precision$ time pyrdp-convert.py ../2021-08_pyrdp-1.1.0/rdp_replay_20210826_12-15-33_512_Stephen215343.pyrdp -f mp4 -o ./
[*] Converting '../2021-08_pyrdp-1.1.0/rdp_replay_20210826_12-15-33_512_Stephen215343.pyrdp' to MP4
100% (1465 of 1465) |##############################################| Elapsed Time: 0:00:42 Time: 0:00:42
[+] Succesfully wrote '/home/olivier/Documents/gosecure/research/projets/2021-10_pyrdp-video-precision/rdp_replay_20210826_12-15-33_512_Stephen215343.mp4'
real 0m46.372s
user 1m13.356s
sys 0m1.752s
with PR:
(venv) olivier@barachois:~/gosecure/research/projets/2021-10_pyrdp-video-precision$ time pyrdp-convert.py ../2021-08_pyrdp-1.1.0/rdp_replay_20210826_12-15-33_512_Stephen215343.pyrdp -f mp4 -o ./
[*] Converting '../2021-08_pyrdp-1.1.0/rdp_replay_20210826_12-15-33_512_Stephen215343.pyrdp' to MP4
100% (1465 of 1465) |##############################################| Elapsed Time: 0:00:46 Time: 0:00:46
[+] Succesfully wrote '/home/olivier/Documents/gosecure/research/projets/2021-10_pyrdp-video-precision/rdp_replay_20210826_12-15-33_512_Stephen215343.mp4'
real 0m49.827s
user 1m18.960s
sys 0m1.620s
larger file size (it's longer):
-rw-r--r-- 1 olivier olivier 5.5M Oct 27 12:00 small-fix-drift.mp4
-rw-r--r-- 1 olivier olivier 4.0M Oct 27 11:59 small-original.mp4
ffprobe on result with PR:
Duration: 00:00:44.64, start: 0.080000, bitrate: 1021 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x1324, 1019 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
ffprobe on result with master:
Duration: 00:00:34.30, start: 0.066667, bitrate: 958 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x1324, 956 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Packet(frame?) count, original:
$ ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 small-original.mp4
1029
with PR:
$ ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 small-fix-drift.mp4
1116
As discussed offline, I think we need to fully revisit how to calculate FPS/insert still frames.
As a starting point, we can know the expected wallclock duration of the recording based on the first PDU time and last PDU time. From there, it should be easy to calculate the total number of frames and make sure that they are dispersed evenly through the recording.
Then PDUs should be snapped to the nearest(?) frame. I'll probably abandon this or push significant changes when I have a chance.