audiocraft
audiocraft copied to clipboard
facebookresearch
Suggestion: Add support for continuing user selected songs with generate_continuation() to app.py
My last issue https://github.com/facebookresearch/audiocraft/issues/36 was about this issue, but it evolved into something else.
This could be hacked together by using the "Melody condition" for the user selected song.
Then adding a gr.Checkbox() to toggle to use generate_continuation() instead of generate_with_chroma() that's normally used with melody.
More or less like this demo: https://github.com/facebookresearch/audiocraft/blob/main/demo.ipynb
# You can also use any audio from a file. Make sure to trim the file if it is too long!
prompt_waveform, prompt_sr = torchaudio.load("./assets/bach.mp3")
prompt_duration = 2
prompt_waveform = prompt_waveform[..., :int(prompt_duration * prompt_sr)]
output = model.generate_continuation(prompt_waveform, prompt_sample_rate=prompt_sr, progress=True)
display_audio(output, sample_rate=32000)
Added it in my fork, the feature is pretty cool actually. It also works with melody conditioning but imo the result is kinda dirty/noisy.
@rkfg Wow! You've been busy since the last issue. Yes, this is exactly what I needed.
Just a little. I merged the stuff from the upstream and removed my deprecated code, then added some useful things like seed management, model unloading and caching, and customizable overlap. The main branch is now dev, it's better than long.
added to my fork as well along with a multiprompt feature. You can continue any song you input with any length to any length you want:
have fun: https://github.com/GrandaddyShmax/audiocraft_plus/tree/plus in the gradio interface you'll find the explanation on how to use each feature
there is also a huggingface version but you'll need to clone the space and use GPU: https://huggingface.co/spaces/GrandaddyShmax/MusicGen_Plus
as well there is a colab: https://colab.research.google.com/github/camenduru/MusicGen-colab/blob/main/MusicGen_ClownOfMadness_plus_colab.ipynb
