[Question] Exporting HifiGanModel and Tacotron2Model for TTS on Android

[inventionsbyhamid]

trafficstars

I am using TTS using the below code. I want to run TTS offline on an Android device as a next step using the same pretrained models.

import soundfile as sf
from nemo.collections.tts.models import HifiGanModel
from nemo.collections.tts.models import Tacotron2Model

spec_generator = Tacotron2Model.restore_from("tts_hi_female_tacotron2.nemo")
spec_generator.eval()

vocoder = HifiGanModel.from_pretrained(model_name="tts_hifigan")

parsed = spec_generator.parse("न्यू यॉर्क में सड़क धंसने से एक विशाल सिंकहोल बनने का वीडियो वायरल हो गया है। इसमें भारी बारिश के बाद इलाके में एक वैन को सिंकहोल में गिरते हुए देखा जा सकता है।")

spectrogram = spec_generator.generate_spectrogram(tokens=parsed)
audio = vocoder.convert_spectrogram_to_audio(spec=spectrogram)

sf.write("speech.wav", audio.to('cpu').detach().numpy()[0], 22050)

I went through the issue How to use/port quartzNet model on mobile? which suggests it is possible to export the model.

Pytorch documentation for exporting model to run on android shows below code

example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(model, example)
traced_script_module_optimized = optimize_for_mobile(traced_script_module)
traced_script_module_optimized._save_for_lite_interpreter("app/src/main/assets/model.ptl")

Where I am stuck now -> What to pass in example as second paramter to torch.jit.trace for tacotron2 and hifigan? I would need to export both tacotron2 and hifigan separately? How to pass output from one model to input to another on android? (This only if I am able to achieve first two steps)

[titu1994]

Please check the export section in Nemo documentation.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale.