rotary-embedding-torch icon indicating copy to clipboard operation
rotary-embedding-torch copied to clipboard

about axial rotary embeddings

Open raindrop313 opened this issue 3 years ago • 0 comments

Hi, Thank you for sharing this code with us. However, I was confused with the axial rotary embeddings in rotary_embedding_torch.py file. " elif freqs_for == 'pixel': freqs = torch.linspace(1., max_freq / 2, dim // 2) * pi " Where does this formula come from?What parameter is max_freqs?Why the freqs is not " 1/(10000^(2i/d))"?

Thank you again.

raindrop313 avatar Jun 03 '22 16:06 raindrop313