transformer-xl
transformer-xl copied to clipboard
feat: replace einsum with matmul for efficiency
Hi @kimiyoung,
I recently started to use transformer-xl on my personal project (mostly based on this repo), and I found the speed can be improved about 3 to 4 times by replacing all the torch.einsum with torch.matmul (equals to @).
Feel free to review it if you have time.