Johann Diep

Reason for combining rotary and relative positional embedding?

1

Hi, Awesome work first of all. Is there a reason why you would combine both rotational as well as relative positional embedding in your Attention class? I would assume one...