audiocraft
audiocraft copied to clipboard
facebookresearch
FR: Sequencing sections Text Prompt
Can we introduce a prompt mechanic that will allow us to feed a "time sequential" theme/feel that would look/act like this:
general prompt text [Section 1 prompt {x}(a) [section 2 prompt {y} (b)] [section 3 prompt {z}(c)]... return to general prompt
Where the different sections are sequential in the time of the composition. Where something like Bohemian Rhapsody would be
Queen rock song 85 bpm [a cappella, harmonies {30}] [piano with vocals {120}] [piano with vocal {120}] [guitar solo {30}] [rock opera with vocals {45}] [105 bpm guitar with vocals {45}]
-
where the values inside the {} are seconds or some specific measure of time
-
but if they are in parentheses instead it is a percentage of time
-
and if no values are specified for a particular bracket then those unspecified are evenly divided against what was already specified
I suspect continuity could be maintained by looping in original generated audio as a Melody to base the remaining off of? With some audio overlap built in to better weave the sounds together.
already exists, have fun: https://github.com/GrandaddyShmax/audiocraft_plus/tree/plus in the gradio interface you'll find the explanation on how the multiprompt works.
there is also a huggingface version but you'll need to clone the space and use GPU: https://huggingface.co/spaces/GrandaddyShmax/MusicGen_Plus
as well there is a colab: https://colab.research.google.com/github/camenduru/MusicGen-colab/blob/main/MusicGen_ClownOfMadness_plus_colab.ipynb
