RTNeural
RTNeural copied to clipboard
jatinchowdhury18
Support for TCN
Hi, I have been using TCNs to train and model more complicated effects like Delay and Reverb (https://colab.research.google.com/corgiredirector?site=https%3A%2F%2Farxiv.org%2Fabs%2F2112.02926)
I have also had some luck in using these in the Neutone plugin, however, it's not realtime. There is a long delay making it unplayable.
Wondering whether it would be possible to support TCN trained models or neutone models directly?
Thanks,
Hello!
Yes, it is definitely possible to implement a TCN model (as described in the paper you linked) within RTNeural. I had done an implementation a few months ago, but haven't pushed it anywhere yet since it hasn't really been tested. For that i remember having to implement a FiLM layer for myself, as well as a TCNBlock which has some non-sequential routing in it. I'm happy to dig up that code and discuss it more if you like!
If you want to have a look at https://github.com/jatinchowdhury18/RTNeural/pull/116, @purefunctor has also implemented a "MicroTCN" network. We should have that PR merged in the next couple of days.
I would be very interested in seeing any code you have for TCN networks, the FiLM layer is something I hadn't even gotten to yet so any help would be amazing. Thank you. In the meantime I'll take a look at 116.
Sure thing! I just uploaded my code here: https://github.com/jatinchowdhury18/RTNeural-Experimental/blob/main/examples/snafx/SNAFxModel.h
As I had mentioned, it's not very well tested, but hopefully the overall ideas there are useful!