maybe a bug in `update_kv` function

Open HarryWu99 opened this issue 1 year ago • 1 comments

https://github.com/FasterDecoding/SnapKV/blob/ea655b18061313e088879bd2b4a3e3c0c2dc2e21/snapkv_utils.py#L50

In update_kv function, instead of using the function's arguments attention_mask, this variable is overridden.

Apr 27 '24 13:04 HarryWu99

Just refactored the codebase. I think the arg is dummy and will remove it in the future update. You can test to see if there is any bug! Thx!

Apr 28 '24 16:04 leeyeehoo