RAVE
RAVE copied to clipboard
acids-ircam
Feature Request: Reinitializing Prior model when running with nn~
Hello,
This is less of an issue but more of a feature request.
I've found that controlling the Prior guided generation in realtime tends to produce either really good results but most of the time, it 'drifts' away into noise, or into a not really usable part of the RAVE latent space.
I've figured out that a more controlled way to seek for 'good' places in the latent space by the Prior can be done by simply re initializing (by reloading your .ts model) in the nn~ object which runs the Prior function.
My intuition is that each time the Prior is reinitialized, it starts with a randomly selected seed, which may or may not produce interesting audio results.
The request would be to add a function in the exported Prior model which one can manually select a seed or to manually reinitialize the model, as a way to 'reset' and select another seed for reproducibility and further control.
Thanks!
Agree! It would be also really neat to be able of maintain the seed, so that we can go back to a certain place of the space.
On Thu, Jul 28, 2022, 11:19 AM Moisés Horta Valenzuela < @.***> wrote:
Hello,
This is less of an issue but more of a feature request.
I've found that controlling the Prior guided generation in realtime tends to produce either really good results but most of the time, it 'drifts' away into noise, or into a not really usable part of the RAVE latent space.
I've figured out that a more controlled way to seek for 'good' places in the latent space by the Prior can be done by simply re initializing (by reloading your .ts model) in the nn~ object which runs the Prior function.
My intuition is that each time the Prior is reinitialized, it starts with a randomly selected seed, which may or may not produce interesting audio results.
The request would be to add a function in the exported Prior model which one can manually select a seed or to manually reinitialize the model, as a way to 'reset' and select another seed.
Thanks!
— Reply to this email directly, view it on GitHub https://github.com/acids-ircam/RAVE/issues/107, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYFRPSZ5TXKRDIUHNSKBDVWKQGXANCNFSM545VP4GA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
One possible way to do this is within the nn tilde library in MaxMSP. The encoder outputs a list of 8 signal values which are then fed to the decoder. These values can be recorded externally, and then played back into the decoder. I believe this would reproduce the same sonic result.
However, i would also love this as a built in feature.
I also will add the caveat that this is my intuition using the nn tilde library, not in direct understanding of the .ts model.
@branlsnyder Yes, indeed I've been experimenting with that approach; recording each individual latent dimension into audio, from the output of either the prior or the encoder model and it seems to work quite well for stuff like 'looping' and 'recording' within latent space.
The feature here would be similar, albeit not the same, since what I'm describing is more to reproduce the actual latent codes generated from the Prior and it's subsequent latent trajectory...
I'm thinking there should be a way to make the 'random seed' an exposed parameter when exporting as a .ts file, but this goes beyond my experience with Pytorch/torchscript. Will dive deeper into it when I have the time.
Report back if you come up with anything! A controllable random seed Parameter would be dope.
torch.Generator are not yet compatible with torchscript unfortunately. Controllable seed with come with a scriptable generator from torch
