KIVI icon indicating copy to clipboard operation
KIVI copied to clipboard

How to understand the code: triton_quantize_and_pack_along_last_dim(value_states_full[:, :, :1, :].contiguous(), self.group_size, self.v_bits)

Open chenyehuang opened this issue 8 months ago • 3 comments

微信截图_20240624190119

I don't understand why the input data is value_states_full[:, :, :1, :].contiguous() instead of value_states_full[:, :, :-1, :].transpose(2, 3).contiguous()

chenyehuang avatar Jun 24 '24 11:06 chenyehuang