equinox icon indicating copy to clipboard operation
equinox copied to clipboard

Published 20 hours ago • verified publisher icon patrick-kidger

RoPE Embeddings

Open Artur-Galstyan opened this issue 8 months ago • 9 comments

Added RoPE embeddings from the RoFormer paper.

I need to add this to my transformer to perform some tests first before I can mark this as ready. Also if it's not needed/wanted, feel free to close this PR :)

If merged, closes #566

Artur-Galstyan avatar Oct 22 '23 12:10 Artur-Galstyan