[FEATURE] Pre-convolution for LSTM

[sdatkinson]

Option to incorporate a convolutional layer in front of LSTM models. Helps w/ sensitivity to delay, and generally improves training.

[38github]

As an LSTM fan I have to ask - how is this going?

[38github]

My (subjective and objective) findings between WaveNet and LSTM is:

WaveNet - Pros

• Achieves good frequency response even in extreme cases like EQs
• Manages to model some ambience
• Easy to train

WaveNet - Cons

• Unbearable higher frequencies that becomes more prominent the more distortion the model has. Sounds like aliasing and in the end all high gain amplifiers/pedals/etc sound about the same.
• Post-ringing
• CPU usage

LSTM - Pros

• Distortion sounds very good and no audible aliasing like sounds.
• No post-ringing
• Low CPU

LSTM - Cons

• Does not achieve very good frequency response with extreme EQing like WaveNet does
• Loses some higher frequencies at lower quality BUT is solved by increasing mostly hidden size but also increases CPU usage
• Hard time handling ambience.
• Very finicky with training parameters