yoyodyne icon indicating copy to clipboard operation
yoyodyne copied to clipboard

Disables `norm_first` in the transformer encoder

Open kylebgorman opened this issue 6 months ago • 0 comments

This allows us to use the experimental, but supposedly faster, nested tensor API:

https://pytorch.org/docs/stable/nested.html

As the documentation indicates, this apparently is particularly helpful with padding.

Closes #214. Mutually exclusive with #225.

kylebgorman avatar Jul 30 '24 15:07 kylebgorman