Allow backward neural connections

[ghost]

I'm trying to use wavenet to implement audio conversion instead of audio generation, which means that I don't have to restrict learning on previous inputs, so that the network would look something like this:

Would there be a way to do this with wavenet, so that we are not restricted to only previous inputs?