Thomas Melistas

No fp16 support from fast-transformers (CausalDotProduct)

75

There is the following `RuntimeError: expected scalar type Float but found Half` when using fp16 (for causal attention). That is because the fast-transformers' `CausalDotProduct` doesn't support fp16. Do you think...

FixNorm alongside ScaleNorm

3

Hello there, Do you think it would be possible to add FixNorm in conjuction with ScaleNorm? In the Transformers without tears paper results it does improve the results and it...

Added Spleeter support

2

[Spleeter](https://github.com/deezer/spleeter) is a source separation library by Deezer, with amazing results. I added it as an option because after hearing MaD Twinnet's output I wasn't that satisfied. But, even though...