VoiceCraft
VoiceCraft copied to clipboard
jasonppy
Usage instructions.
(really really impressed by the demo, so much further than the best SOTA model I found so far, congrats on the great work).
Running with docker/jupyter.
I followed the Docker/jupyter instructions, to the letter (I'm not at all familiar with jupyter, very with docker).
It went mostly well.
I keep running cells/advancing, again and again, until I get at the bottom.
And then nothing? What's supposed to happen? I don't see any new instructions, no new files, anything, I'm fairly lost.
Running as a script.
The jupyter stuff is great to get to know the project, but (unless I don't understand what jupyter is), it won't really help getting voicecraft integrated into my project / enable me to generate thousands of files / "call" voicecraft programmatically from my nodejs system.
In other, there is something like:
python3 voicecraft/bin/inference.py --text="Read this text" --model_path="voicecraft/model/file.something" --voice_sample="/tmp/voices/robert.wav" --output="/tmp/sample_voicecraft_output.wav" --device=cpu
What's the equivalent for voicecraft, and how do I get to the point where it'll agree to run? (running inside docker is fine, or outside docker too, just need to get it to run).
I found main.py, and I think the options for the command line are in config.py, but I don't know which options I need and which I don't / I don't know how to use the script. I didn't find an example of how to use it, but I'll keep looking.
Intonation.
I might be getting a bit ahead of myself here since I don't have it running yet, but maybe you know: will intonation/style transfer through? Like if my voice sample has the person whispering, will the output be whispering? Same for shouting, crying, etc. That's really the big thing missing from my system, is there any way to get that to work with voicecraft, do you know?
Thanks a lot in advance! Cheers.
Thanks!
about supporting. I appreciate that you like the demo and want to incorporate voicecraft into your nodejs system. It shouldn't be hard to wrap what's in the jupyter notebook into a python script that you mentioned so you can call from the command line.
About intonation. The model will clone whatever tone the prompt uses so please give it try.
@jasonppy thanks for the reply. I found a PR with such a script (https://github.com/jasonppy/VoiceCraft/pull/34), do you have any idea when it will be tested/merged?
Also, you didn't answer my question about the jupyter script: it's not doing anything, I just get to the bottom and nothing happens, what am I doing wrong / what's going on there?
Thanks!
This Jupyter stuff is just another layer of unnecessary complexity that induces further points of failure and it is annoying to work with versus actual shell and real python scripts.
I can get conda to work in docker bash, but from the Notebook it complains "CondaError: Run 'conda init' before 'conda activate'". I dumped the bash variables with set and they are not identical, although the correct files are sourced. Fix was to edit the bashrc file and remove the "if not interactive don't do anything stuff" at the top... Crazy, why is this even there. This messes up all sorts of things from cronjobs to mailserver daemon.
Anyway, then when downloading the model from Jupyter, Firefox ate up over 30GB of system memory and it crashed my computer that "only" has 32GB ... now it is at 28GB again, I hope it will make it this time. Edit: again, I fixed this by copy&pasting as much as possible in actual bash console, and then I could run the demo. Like 90% of the Jupyter notebook was just failure after failure and using docker bash console to fix. Also don't trust the Notebook. You have to stick to the README.md to install correct dependencies via conda/pip in bash console, and what's in the inference_tts.ipynb is incomplete/incorrect although it looks the same. You also need to install various stuff manually that is not mentioned, such as wget and certain python libraries. It is very bad to work in the Notebook. It easily goes unnoticed when a subcommand fails. And if you misclick then it runs in the background and you can't un-hide the output and stuff like this, like it is just another bug (you can still check output in docker logs).
This Jupyter stuff is really bad, you really should get rid of it as soon as possible. Just gives everyone a hard time and bugs stuff out for no reason.
@ballerburg9005 :
I do agree about Jupyter.
I think it's great for researchers who want to understand how this works, but not really great for people who need to "use" the project.
And I think enabling more people to use the project easier is good for a project like this, it will bring in more eyes on the code, more bug reports, more contributions etc.
And I'm not complaining or saying anything bad about the project or author(s), this is really amazing work, and it's incredibly cool that we are allowed to use it. This is bleeding edge / state of the art, and that is expected to come with some rough edges / not be the easiest to work with out of the box. This is stuff that gets better with time. I just hope "ease of use" gets better soon, that would be great for me and (I think) lots of others.
The number of thumbs up on the above comment I think mean lots of people want an easy way to use this (awesome) tool.
I can't "use" the Jupyter thing from the command line, I can't "call" it from other code/projects that are building on/around this project, etc.
Somebody did a PR with a normal usage/inference script like other projects have, I'll be testing that soon. But the person who wrote it said they haven't tested it (which is weird for such a long script...), so I'm not sure how well that will go.
