manim-voiceover
manim-voiceover copied to clipboard
ManimCommunity
Using RecorderService manim-voiceover hangs (before or after releasing the 'r' key?)
Description of bug / unexpected behavior
I render a file with CoquiService and both manim and manim-voicover work correctly. Then I pass to RecorderService, I select the input device (e.g. 13 - default) and start recording pressing the 'r' key. When I release the 'r' key manim-voiceover hangs as if lost in an infinite idle cycle. In fact, inspecting folder ./media/voiceovers I see that no file has been produced, therefore I suspect that manim-voiceover hangs waiting the user to press 'r'...
Expected behavior
Manim-voiceover should start recording the voiceover as soon as the user presses 'r'. After releasing the 'r' key manim-voiceover should ask to choose from the following options:
-
l to [l]isten to the recording
-
r to [r]e-record
-
a to [a]ccept the recording
How to reproduce the issue
Code for reproducing the problem
from manim import *
from manim_voiceover import VoiceoverScene
# from manim_voiceover.services.gtts import GTTSService
# from manim_voiceover.services.coqui import CoquiService
from manim_voiceover.services.recorder import RecorderService
from math import *
class RecordVoiceover(VoiceoverScene):
def construct(self):
circle = Circle()
# self.set_speech_service(GTTSService(lang='it',tld='it',transcription_model='base'))
self.set_speech_service(RecorderService())
# self.set_speech_service(CoquiService(
# model_name='tts_models/it/mai_male/glow-tts',
# transcription_model='base'
# )
# )
with self.voiceover(
text='''Ora creo un <bookmark mark="A"/> cerchio,
poi lo muovo a <bookmark mark="B"/> destra
e infine lo <bookmark mark="C"/> elimino.
''') as tracker:
self.wait_until_bookmark('A')
self.play(Create(circle,run_time=0.5))
self.wait_until_bookmark('B')
self.play(circle.animate.shift(RIGHT),run_time=0.5)
self.wait_until_bookmark('C')
self.play(FadeOut(circle),run_time=0.5)
Additional media files
Images/GIFs
Logs
Terminal output
PASTE HERE OR PROVIDE LINK TO https://pastebin.com/ OR SIMILAR
System specifications
System Details
- OS (with version, e.g., Windows 10 v2004 or macOS 10.15 (Catalina)): Linux Ubuntu 23.04
- RAM: 32 GB
- Python version (
python/py/python3 --version): 3.10.11 - Installed modules (provide output from
pip list):
Package Version
------------------------------ ------------
accelerate 0.19.0
aiohttp 3.8.4
aiosignal 1.3.1
anyascii 0.3.2
appdirs 1.4.4
async-timeout 4.0.2
attrs 23.1.0
audioread 3.0.0
azure-cognitiveservices-speech 1.29.0
Babel 2.12.1
backports.cached-property 1.0.2
bangla 0.0.2
blinker 1.6.2
bnnumerizer 0.0.2
bnunicodenormalizer 0.1.1
boltons 23.0.0
brotlipy 0.7.0
build 0.10.0
CacheControl 0.12.11
certifi 2023.5.7
cffi 1.15.1
charset-normalizer 3.1.0
clean-fid 0.1.35
cleo 2.0.1
click 8.1.3
click-default-group 1.2.2
clip-anytorch 2.5.2
cloup 0.13.1
cmake 3.26.3
colorama 0.4.6
colour 0.1.5
contourpy 1.0.7
coqpit 0.0.17
crashtest 0.4.1
cryptography 41.0.1
cycler 0.11.0
Cython 0.29.28
dataclasses 0.8
dateparser 1.1.8
decorator 5.1.1
deepl 1.14.0
distlib 0.3.6
docker-pycreds 0.4.0
docopt 0.6.2
dulwich 0.21.5
einops 0.6.1
evdev 1.6.1
ffmpeg-python 0.2.0
filelock 3.12.0
Flask 2.3.2
fonttools 4.39.4
frozenlist 1.3.3
fsspec 2023.5.0
ftfy 6.1.1
future 0.18.3
g2pkk 0.1.2
gitdb 4.0.10
GitPython 3.1.31
glcontext 2.3.7
gruut 2.2.3
gruut-ipa 0.13.0
gruut-lang-de 2.0.0
gruut-lang-en 2.0.0
gruut-lang-es 2.0.0
gruut-lang-fr 2.0.2
gTTS 2.3.2
html5lib 1.1
huggingface-hub 0.15.1
idna 3.4
imageio 2.31.0
importlib-metadata 6.6.0
importlib-resources 5.12.0
inflect 5.6.0
installer 0.7.0
isosurfaces 0.1.0
itsdangerous 2.1.2
jamo 0.4.1
jaraco.classes 3.2.3
jeepney 0.8.0
jieba 0.42.1
Jinja2 3.1.2
joblib 1.2.0
jsonlines 1.2.0
jsonmerge 1.9.0
jsonschema 4.17.3
k-diffusion 0.0.15
keyring 23.13.1
kiwisolver 1.4.4
kornia 0.6.12
lazy_loader 0.2
librosa 0.10.0.post2
lit 16.0.5.post0
llvmlite 0.39.1
lockfile 0.12.2
manim 0.17.3
manim-voiceover 0.3.3.post0
ManimPango 0.4.3
mapbox-earcut 1.0.0
markdown-it-py 2.2.0
MarkupSafe 2.1.3
matplotlib 3.7.1
mdurl 0.1.0
mecab-python3 1.0.5
moderngl 5.8.2
moderngl-window 2.4.1
more-itertools 9.1.0
mpmath 1.3.0
msgpack 1.0.5
multidict 6.0.4
multipledispatch 0.6.0
mutagen 1.46.0
networkx 2.8.8
nltk 3.8.1
num2words 0.5.12
numba 0.56.4
numpy 1.23.5
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-cupti-cu11 11.7.101
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
nvidia-cufft-cu11 10.9.0.58
nvidia-curand-cu11 10.2.10.91
nvidia-cusolver-cu11 11.4.0.1
nvidia-cusparse-cu11 11.7.4.91
nvidia-nccl-cu11 2.14.3
nvidia-nvtx-cu11 11.7.91
openai-whisper 20230314
packaging 23.1
pandas 2.0.2
pathtools 0.1.2
pexpect 4.8.0
Pillow 9.5.0
pip 23.1.2
pkginfo 1.9.6
pkgutil_resolve_name 1.3.10
platformdirs 3.5.1
poetry 1.5.1
poetry-core 1.6.1
poetry-plugin-export 1.4.0
pooch 1.6.0
protobuf 3.19.6
psutil 5.9.5
ptyprocess 0.7.0
PyAudio 0.2.13
pycairo 1.23.0
pycparser 2.21
pydub 0.25.1
pyglet 1.5.27
Pygments 2.15.1
pynndescent 0.5.10
pynput 1.7.6
pyOpenSSL 23.2.0
pyparsing 3.0.9
pypinyin 0.49.0
pyproject_hooks 1.0.0
pyrr 0.10.3
pyrsistent 0.19.3
pysbd 0.3.4
PySocks 1.7.1
python-crfsuite 0.9.9
python-dateutil 2.8.2
python-dotenv 0.21.1
python-slugify 8.0.1
python-xlib 0.33
pyttsx3 2.90
pytz 2023.3
PyWavelets 1.4.1
PyYAML 6.0
rapidfuzz 2.15.1
regex 2023.6.3
requests 2.31.0
requests-toolbelt 1.0.0
resize-right 0.0.2
rich 13.4.1
scikit-image 0.21.0
scikit-learn 1.2.2
scipy 1.10.1
screeninfo 0.8.1
SecretStorage 3.3.3
sentry-sdk 1.25.0
setproctitle 1.3.2
setuptools 67.7.2
shellingham 1.5.1
six 1.16.0
skia-pathops 0.7.4
smmap 5.0.0
soundfile 0.12.1
sox 1.4.1
soxr 0.3.5
srt 3.5.2
stable-ts 2.6.2
svgelements 1.9.5
sympy 1.12
tensorboardX 2.6
text-unidecode 1.3
threadpoolctl 3.1.0
tifffile 2023.4.12
tiktoken 0.3.1
tokenizers 0.13.3
tomli 2.0.1
tomlkit 0.11.8
torch 2.0.1
torchaudio 2.0.2
torchdiffeq 0.2.3
torchsde 0.2.5
torchvision 0.15.2
tqdm 4.65.0
trainer 0.0.20
trampoline 0.1.2
transformers 4.29.2
triton 2.0.0
trove-classifiers 2023.5.24
TTS 0.14.3
typing_extensions 4.6.3
tzdata 2023.3
tzlocal 5.0.1
umap-learn 0.5.1
unidic-lite 1.0.8
urllib3 1.26.15
virtualenv 20.23.0
wandb 0.15.4
watchdog 2.2.1
wcwidth 0.2.6
webencodings 0.5.1
Werkzeug 2.3.4
wheel 0.40.0
yarl 1.9.2
zipp 3.15.0
LaTeX details
- LaTeX distribution (e.g. TeX Live 2020): TeX Live 2022/Debian
- Installed LaTeX packages:
FFMPEG
Output of ffmpeg -version:
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7.3.0 (crosstool-NG 1.23.0.449-a04d0)
configuration: --prefix=/tmp/build/80754af9/ffmpeg_1587154242452/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho --cc=/tmp/build/80754af9/ffmpeg_1587154242452/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc --disable-doc --enable-avresample --enable-gmp --enable-hardcoded-tables --enable-libfreetype --enable-libvpx --enable-pthreads --enable-libopus --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-static --enable-version3 --enable-zlib --enable-libmp3lame --disable-nonfree --enable-gpl --enable-gnutls --disable-openssl --enable-libopenh264 --enable-libx264
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Additional comments
I can't reproduce this locally. Can you maybe insert some breakpoints (import ipdb; ipdb.set_trace()) to manim-voiceover source locally and tell me which line causes the hang?
Well, doing like you asked I determined that manim-voiceover hangs in services/recorder/utility.py between line 160 (reached) and line 163 (never reached). It reaches line 179, therefore it seems that MyListener() is not able to recognise pression of 'r' key. i.e. that self.listener.key_pressed is always false!
Doing more debugging I determined that MyListener.on_press() is never called (the listener gets initialised and started, though), therefore there is no chance for key_pressed to become True, and its value remains None! Of course, I don't understand the reason for this behaviour...
Ok, I found the origin of the issue: the fact is that pynput is not meant to work on Wayland but only on Xorg. When I switched to Ubuntu on Xorg all worked as expected.
Knowing this, you should either search for an alternative to pynput that works also on Wayland (such as this which, unfortunately, is currently unmaintained), or warn users to avoid Wayland when using manim-voiceover on Ubuntu...
Then this might relate to #44, you can follow the discussion there. (Note that I had issues with Gradio, and that's why I didn't merge until now) The CLI based approach was a quick hack to get the MVP going, and the ideal solution would be a standalone UI that works without cross platform compatilibility issues.
I found the following options:
https://github.com/PySimpleGUI/PySimpleGUI https://github.com/hoffstadt/DearPyGui https://github.com/beeware/toga
We could also go for an Electron app or locally hosted web app like Jupyter Notebook but then it would be more complicated to ship Python and JS code in the same package, albeit more future-proof. Open to suggestions. cc @o-alexandre-felipe