tvm icon indicating copy to clipboard operation
tvm copied to clipboard

[Relax] Fix llama4_rope_with_position_map to support partial rotary factor

Open guan404ming opened this issue 1 month ago • 2 comments

Related Issue

closes #17715

Why

  • Phi-4 uses: partial_rotary_factor = 0.75 (rotary_dim = 96) + longrope scaling
  • Longrope requires: Both long_factors + short_factors packed into one buffer
  • Expected buffer size: (rotary_dim,) = (96,) total
    • First half [0:48] = long_factors
    • Second half [48:96] = short_factors
  • llama4_rope_with_position_map still had old size (rotary_dim // 2,) = (48,)

guan404ming avatar Nov 28 '25 04:11 guan404ming

cc @tlopex @mshr-h

guan404ming avatar Nov 28 '25 07:11 guan404ming

cc @MasterJH5574

tlopex avatar Nov 28 '25 07:11 tlopex