piper
piper copied to clipboard
rhasspy
GLaDOS voice
Is there a GLaDOS voice for pipers as it was for larynx (https://github.com/rhasspy/larynx/issues/56)? Or possibly an easy way to convert one to another? I added phonemes and missing entries in the json file, but still there are phonemes missing and errors about the model
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: scales for the following indices
index: 0 Got: 3 Expected: 2
Please fix either the inputs or the model.
Tomorrow I will train a Glados dataset, but what worries me is the license to publish it.
That's what I was afraid of. Would instructions to train a dataset on one's own be more in the clear? I have no idea about hardware requirements, though.
To make things easier, I use colab notebooks, since I don't have the hardware. To run it locally, you would need an NVIDIA GPU and the parameters (eg batch_size) can be run according to the capabilities of your GPU.
I don't have the hardware either. And I guess if detailed instructions were published, that could still get DMCA'd as did tools like yt-dl
@rmcpantoja any update on the Glados training?
Hi @dnhkng, I have two GLaDos models made, one in Spanish and the other in English through my colab notebooks, but unfortunately since they are datasets with a lot of corpus, they require more training and I do not have the resources to buy colab pro. They are the following: English and Spanish
@rmcpantoja The English link doesn't work. I was going to try a finetune on the original game voice data. I have 2x 4090s, so I should have enough compute.
I could rip the voices from https://theportalwiki.com/wiki/GLaDOS_voice_lines but is there a dataset with this already prepared? Happy to share the results!
Hi @dnhkng, It sounds strange, I am able to open the Drive folder with the English model without problems. Anyway, here I've a model exported to onnx
The model was trained using this dataset, but I was in charge of fixing many incorrect transcriptions.
@rmcpantoja Thanks for the export! I found the checkpoint file eventually though, sorry! Sounds pretty good, I see it trained on colab for 2.25 hours.
I scrapped the GLaDOS dataset (only using the Portal 2 voice and DLC), manually filtered out all the wav files that contained extras (Laughing, telephones, beeping, etc), and also fixed all the text. That gave me about 1 hour of high-quality data. I have currently fine-tuned for 15 hours on a 4090, and it sounds very good, and the loss is still decreasing. I will train for 24 hours, and see how the loss curves look.
EDIT: Here is a samples after 24H of finetuning. 'a' is the generated sample, 'b' is an unseen sample from the the game. https://drive.google.com/drive/folders/1WVpS2zlJ9JqXIYV8Fkjoy5Fjz-eWPaEh?usp=sharing
I think the generated sample is better! Kudos to Piper, this is amazing!
@dnhkng Hello is it possible to get the model ?
Yes, I will share it in the next few days. Doing a big refactor on the inference code.
OK, the model is available here: https://github.com/dnhkng/GlaDOS
You can find the GlaDOS model in the models directory.
It includes my new code base to use the voice. Have a look in the Jupyter Notebook on how to use it.
If you instead want to use it with Piper, just take a medium size model, and copy the .onnx.json file, and rename it as glados.onnx.json, and it will run with Piper.
For those of you who want to run GlaDOS onnx model on iOS, Android, Raspberry Pi, or use C++, C, Go, C#, Kotlin, Swift, Python, Java, or on Windows, Linux, macOS, etc, please have a look at https://github.com/k2-fsa/sherpa-onnx
We provide a colab to show you how to convert the GlaDOS model to sherpa-onnx https://colab.research.google.com/drive/1m3Zr8H1RJaoZu4Y7hpQlav5vhtw3A513?usp=sharing
The following is a sample command using the converted model with sherpa-onnx
# You can also use sherpa-onnx-offline-tts-play
sherpa-onnx-offline-tts \
--vits-model=./glados.onnx \
--vits-tokens=./tokens.txt \
--vits-data-dir=./espeak-ng-data \
--output-filename=./test-glados.wav \
"How are you doing? This is a text-to-speech application using next generation Kaldi."
https://github.com/rhasspy/piper/assets/5284924/9e75eca4-b73d-46a4-b0c4-e88df3a2ae4b
By the way, I just managed to build Android APKs for the pre-trained GLaDOS models mentioned in this issue, i.e, for the following two models:
- A Spanish model from https://drive.google.com/file/d/12tNCCyd0Hf5jsyqCw8828kLSHHx5LOw9/view?usp=drive_link
- An English model from https://github.com/dnhkng/GlaDOS
You can find the APKs at https://k2-fsa.github.io/sherpa/onnx/tts/apk.html
For your convenience, the download address is given below:
- sherpa-onnx-1.9.3-arm64-v8a-en-tts-vits-piper-en_US-glados.apk
- sherpa-onnx-1.9.3-arm64-v8a-es-tts-vits-piper-es-glados-medium.apk
If you are interested in how we build the APK, please read the following documentation https://k2-fsa.github.io/sherpa/onnx/android/index.html
You can also try the models in the following huggingface space in your browser
http://huggingface.co/spaces/k2-fsa/text-to-speech
@LaneaLucy I would, but I'm not sure how to do it! I trained using the voice from the game. Is there a German voice in the German edition?
(I live in Bavaria, which is almost German :) )
@LaneaLucy I am doing new voice training soon, join my discord https://discord.com/invite/ERTDKwpjNB and in the general section I'm discussing it now.
@dnhkng Your GlaDos model is amazing! You should also create a Cave Johnson and Wheatley model. Can you imagine having multiple devices speaking to each other with the different voices? 🤣 https://theportalwiki.com/wiki/Cave_Johnson_voice_lines https://theportalwiki.com/wiki/Wheatley_voice_lines
It's not clear to me what I have to add to which directories in the Piper data folder. I don't see any large file with extension ONNX in that repository. What gives?
FYI the model files can now be found on the releases page of that project. Also this guide was very helpful helpful in adding a custom voice to Piper running in HA
FYI the model files can now be found on the releases page of that project. Also this guide was very helpful helpful in adding a custom voice to Piper running in HA
This release page seems to be missing the .json file
@maxi1134
FYI the model files can now be found on the releases page of that project. Also this guide was very helpful helpful in adding a custom voice to Piper running in HA
This release page seems to be missing the .json file
Did you find a .json file to use with this?
FYI the model files can now be found on the releases page of that project. Also this guide was very helpful helpful in adding a custom voice to Piper running in HA
This release page seems to be missing the .json file
Did you find a .json file to use with this?
I just quickly searched the repo and found a json file in models/TTS/glados.json that looks similar to what piper expects: https://github.com/dnhkng/GLaDOS/blob/main/models/TTS/glados.json
@dnhkng Your GlaDos model is amazing! You should also create a Cave Johnson and Wheatley model. Can you imagine having multiple devices speaking to each other with the different voices? 🤣 https://theportalwiki.com/wiki/Cave_Johnson_voice_lines https://theportalwiki.com/wiki/Wheatley_voice_lines
FYI: I just trained the German glados voice the recent days and it is quite promising. I plan to release it (on hugggingface), including my toolchain and the "training data" (well, rather how to extract the voice lines from the game and my selection on "good" samples).
https://github.com/user-attachments/assets/e4b7f92b-cc7c-40d0-95b9-c21c35acaf43
Edit: Here is my trained model: https://huggingface.co/systemofapwne/piper-de-glados