LongICLBench icon indicating copy to clipboard operation
LongICLBench copied to clipboard

Published 20 hours ago • verified publisher icon TIGER-AI-Lab

Qwen1.5 seems not using NTK

Open PengWenChen opened this issue 1 month ago • 0 comments

Hi~ Thanks for sharing the evaluation code! I would like to ask where did Qwen1.5 add ntk-interpolation? I traced the code with Qwen2 with huggingface transformers, and found they remove the ntk-interpolation (Qwen1 has ntk but Qwen2/1.5 don't have)

Qwen1.5 should only use RoPE, right?

PengWenChen avatar May 13 '24 03:05 PengWenChen