flashinfer icon indicating copy to clipboard operation
flashinfer copied to clipboard
verified publisher icon flashinfer-ai

[Feature] Llama3.1 RoPE on the fly

Open turboderp opened this issue 1 year ago • 0 comments

Are there any plans to add more options for pos_encoding_mode? Currently "LLAMA" works for Llama3.1+ models but the embeddings are subtly incorrect and accuracy suffers a bit.

turboderp avatar Jan 21 '25 12:01 turboderp