mamedev
Switch to lossless compression for AVI
For years mame has beed recording AVI with audio and video uncompressed. Code-wise, it is sane to use the screenshot buffer and just dump it to video file, but for a user, it feels dirty and generates an overly huge file, let alone the fact that every major emulator out there has some way of compressing the video it records (lack of audio compression doesn't give so much overhead, so everybody still uses uncompressed wav).
There are various ways to solve this.
- The first one, that's now considered too old and ugly (and Windows only), is using VideoForWindows, that gives you the list of codecs it could use. Implemented in FCEUX, Desmume, snes9x.
- The second one is using a built-in codec, like ZMBV or GZip. It is better, because it's fully portable and equally stable for all platforms. Emulators that use it: DOSBox, openMSX, lsnes, Mesen (since the latest commit).
- The third option is embedding an ffmpeg recorder. This requires some tricks to get it to work when one is encoding the recorded video, but is also cross-platform and feature-rich. Emulators that use it: Dolphin, PPSSPP.
Can you propose a container format that provides support for both on-the-fly resolution changes as well as frame rate changes?
Please try to understand our perspective: While an emulator targeting one specific console can "get away with" a lot of things, an emulator like MAME which must support nearly all use cases doesn't have such luxury.
Quite a lot of systems in MAME can both change their refresh rate and their resolution at runtime. While the resolution changes can somewhat trivially be handled by always upscaling everything to a common resolution, this still leaves the question of arbitrary refresh-rate changes. By its very nature, this would (at a minimum) require writing to a separate AVI or video file, as the vast majority (all, maybe?) of the video codecs out there do not support mid-stream changes in frame rate. How would you solve this problem?
This is what Bizhawk and some other emulators do: once the resolution changes, they start dumping to a new file. Some of them also append the resolution to the file name to make it easier to manage if the resolution changes rapidly, like on PSX. This allows for a pixel perfect video in every segment, so not a single pixel of info is lost to resizing, leaving that up to the user. But appending resolution to the file name might make it harder to import such files as a single segment using scripts like avisynth, so it's not required.
Changing framerate on the fly is called variable frame rate, Dolphin does it. But it has to, because the games don't run at constant frame rate at all. And the resulting video is harder to edit, because the editing tools prefer constant frame rate. So if it's not floating framerate, but constant framerate changing every once in a while, you either duplicate frames (using the video framerate that's even to all the framerates used by the game), or also split AVI into segments. OpenMSX uses to have changing framerate in some weird cases, and their solution is also to split AVI.
Splitting the video on resolution changes is the preferred way to handle it, because no video container or format can handle a resolution change during the middle of recording or playback. If you want to avoid splitting the video, then you could force everything into a single resolution when dumping the video. However, this can cause scaling issues from what is displayed and what is being output to the video file.
As for the variable framerate, what Dolphin does is grab the current cycle count, compare it to the previous cycle count, and then place the frame at the correct time. The frame gets placed and adjusted by ffmpeg to hit the target framerate of the video (either 50 or 60fps depending on region).
To see how we do it in code, here is the relevant portion of code for Dolphin which adds the frame to the video: https://github.com/dolphin-emu/dolphin/blob/master/Source/Core/VideoCommon/AVIDump.cpp#L284
mpeg can change of resolution every frame. There's no lossless mode afaict though.
OG.
On Fri, Apr 21, 2017 at 5:37 PM, Chris Burgener [email protected] wrote:
Splitting the video on resolution changes is the preferred way to handle it, because no video container or format can handle a resolution change during the middle of recording or playback. If you want to avoid splitting the video, then you could force everything into a single resolution when dumping the video. However, this can cause scaling issues from what is displayed and what is being output to the video file.
As for the variable framerate, what Dolphin does is grab the current cycle count, compare it to the previous cycle count, and then place the frame at the correct time. The frame gets placed and adjusted by ffmpeg to hit the target framerate of the video (either 50 or 60fps depending on region).
To see how we do it in code, here is the relevant portion of code for Dolphin which adds the frame to the video: https://github.com/dolphin- emu/dolphin/blob/master/Source/Core/VideoCommon/AVIDump.cpp#L284
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mamedev/mame/issues/1894#issuecomment-296225270, or mute the thread https://github.com/notifications/unsubscribe-auth/AI0i8TmNw78ymImk7aiBQlDFfKV1UEZ1ks5ryM1GgaJpZM4LYVrZ .
Lately I forked a project that implements AVI dumper using Camstudio Lossless Codec. In my fork I made it capable of starting a new AVI segment when new resolution or new framerate arrives from the caller. It also splits on 2GB not to cause problems with VfW. It's a part of a bigger project, but that is also GPL, so can be used by MAME of needed.
https://github.com/vadosnaprimer/jpcrr/blob/no-sdl/streamtools/outputs/cscd-control.cpp https://github.com/vadosnaprimer/jpcrr/blob/no-sdl/streamtools/outputs/cscd.cpp https://github.com/vadosnaprimer/jpcrr/blob/no-sdl/streamtools/outputs/cscd.hpp
Variable framerates are generally a pain to encode and upload to streaming sites, so I'd really advice to use segmented AVI instead of something only ffmpeg can read.
There's yet another option: using x264 lossless rgb (may be hard to seek through the video if AVI container is used tho).
If it's used as an external program, video can be piped over to it like prboom+ does it: https://github.com/coelckers/prboom-plus/blob/master/prboom2/src/i_capture.c#L495 https://github.com/coelckers/prboom-plus/blob/master/prboom2/src/i_capture.c#L554
Here's the lossless command line that works for prboom:
x264 -o output.mp4 --qp 0 --muxer mp4 --demuxer raw --input-csp rgb --output-csp rgb --input-res %wx%h --fps 35 -
But using external apps, one can opt to use ffmpeg instead, with its infinite encoding options, which can use all other encoders for video, audio, and can even mux to wide variety of containers from avi to webm. Here's a lossless avi command for ffmpeg:
ffmpeg -c:a pcm_s16le -c:v ffv1 -pix_fmt bgr0 -level 1 -g 1 -f avi
However all this only seems to apply when piping is used, and AFAIK MAME wants all the dependencies to be embedded, and building ffmpeg for that purpose is really complicated. Building x264 is way easier, and probably piping won't be required if it's embedded, but I don't have any knowledge in this field. Yet x264 only provides very limited options, like it can't handle audio or mux the final encode.
Example of ffmpeg+nut piping: https://github.com/clementgallet/libTAS/blob/master/src/library/encoding/AVEncoder.cpp
There's yet another option: using x264 lossless rgb (may be hard to seek through the video if AVI container is used tho).
If it's used as an external program, video can be piped over to it like prboom+ does it: https://github.com/coelckers/prboom-plus/blob/master/prboom2/src/i_capture.c#L495 https://github.com/coelckers/prboom-plus/blob/master/prboom2/src/i_capture.c#L554
Here's the lossless command line that works for prboom:
x264 -o output.mp4 --qp 0 --muxer mp4 --demuxer raw --input-csp rgb --output-csp rgb --input-res %wx%h --fps 35 -But using external apps, one can opt to use ffmpeg instead, with its infinite encoding options, which can use all other encoders for video, audio, and can even mux to wide variety of containers from avi to webm. Here's a lossless avi command for ffmpeg:
ffmpeg -c:a pcm_s16le -c:v ffv1 -pix_fmt bgr0 -level 1 -g 1 -f aviHowever all this only seems to apply when piping is used, and AFAIK MAME wants all the dependencies to be embedded, and building ffmpeg for that purpose is really complicated. Building x264 is way easier, and probably piping won't be required if it's embedded, but I don't have any knowledge in this field. Yet x264 only provides very limited options, like it can't handle audio or mux the final encode.
Example of ffmpeg+nut piping: https://github.com/clementgallet/libTAS/blob/master/src/library/encoding/AVEncoder.cpp
I've managed to get H264 mp4 video recording working in my latest master. I set up a dedicated thread for the encoder and also DMA (i.e. no memory copying), and modified video.cpp to render directly to the buffer used my ffmpeg. It works, but it might need a bit of tidying (due to LLM bloat) given it was AI assisted coding. I'll see if I can put it into a draft pull request.
Meanwhile we noticed that using H264 in an AVI container is likely to have seeking issues: going backward may show frames out of order. In MP4/MKV it's fine, but if WINAPI/VFW is used for video editing (like avisynth), ffmpeg-based importers are required and they may be annoying to use.
A much more reliable lossless codec that works flawlessly in AVI and seeks very fast is UtVideo. It has a VFW decoder as well.