manim icon indicating copy to clipboard operation
manim copied to clipboard

Make combination of video + audio streams more robust; fix incompatibility with `pyav >= 13.0.0`

Open behackl opened this issue 1 year ago • 1 comments

This rewrites the remuxing and combination of the video + audio streams that have been produced throughout Scene.render, which is happening in the SceneFileWriter.combine_to_movie method. The rewritten version now works (at least locally) for pyav >= 12.0.0, while the previous version apparently only worked for pyav >= 12.0.0, pyav < 13.0.0.

As the RTD render images apparently predominantly had a more recent release of pyav, this is what broke the docbuilld for recent builds.

The rewritten version

  • differs from the old one primarily in the way how audio streams are added to the final output container. In cases where we target mp4 output, instantiating the stream directly from the stream given in the auxiliary audio file broke rendering, so now we instantiate it by passing codec name and sample rate explicitly.
  • additionally supports cases where audio files with multiple audio channels are created.

behackl avatar Oct 13 '24 14:10 behackl

I've been debugging this for a bit longer now. While I did successfully fix the verison issue outside of the RTD build container, I can still reproduce it consistently inside of the container. I don't see any obvious differences between the setups any more; it would probably be a good idea to consult our colleagues from pyav and see whether they can tell us more about this.

I'll prepare a minimal example when I have some more time. For now, if anyone else wants to give this a shot; here is my container setup:

$ docker run --rm -it -v /home/behackl/code/manim:/home/docs/manim --user root readthedocs/build:ubuntu-22.04-2024.01.29 bash
# apt install libpango1.0-dev graphviz
# asdf plugin-add python
# su docs
$ bash
$ asdf install python 3.11.9
$ asdf global python 3.11.9
$ cd manim
$ pip install --upgrade --no-cache-dir pip setuptools sphinx
$ pip install --exists-action=w --no-cache-dir -r docs/rtd-requirements.txt
$ pip install --exists-action=w --no-cache-dir -r docs/requirements.txt
$ pip install --upgrade --upgrade-strategy only-if-needed --no-cache-dir .
$ cd .. && mkdir debug && cp manim/example_scenes/debug.py debug/debug.py
$ cp manim/docs/source/_static/click.wav debug/
$ cd debug
$ manim debug.py

where I have prepared the file debug.py in my example_scenes directory containing

import av

from manim import *

av.logging.set_level(av.logging.DEBUG)
config.verbosity = "DEBUG"

class SoundExample(Scene):
    def construct(self):
        dot = Dot().set_color(GREEN)
        self.add_sound("click.wav")
        self.add(dot)
        self.wait()
        self.add_sound("click.wav")
        dot.set_color(BLUE)
        self.wait()
        self.add_sound("click.wav")
        dot.set_color(RED)
        self.wait()

behackl avatar Oct 13 '24 19:10 behackl

Superseded by #3956.

behackl avatar Oct 14 '24 08:10 behackl