Retrieval_Head

Retrieval_Head copied to clipboard

Question about blocking an attention head

1

https://github.com/nightdessert/Retrieval_Head/blob/3ac171a6f71ce7ef1cda57d4215c390fb6ab51f2/faiss_attn/source/modeling_llama.py#L685-L689 Hi there, I'm working on the interpretability of attention heads and your work is really inspiring. I'm not pro in modifications on these attention heads, so I'm a little...

cnlnpjhsy

Errors with Phi3 models and how to run the code with the model Yi-6B-200K?

1

Hi, thank you for the awsome code! When I run `python retrieval_head_detection.py --model_path microsoft/Phi-3-mini-128k-instruct --s 0 --e 50000`, I got the following errors: Could you share the tips to fix...

HayeonLee

unexpected keyword argument 'block_list'

3

python3 needle_in_haystack_with_mask.py TypeError: Qwen2ForCausalLM.forward() got an unexpected keyword argument 'block_list'

yuhuanyuan

https://github.com/nightdessert/Retrieval_Head/blob/3ac171a6f71ce7ef1cda57d4215c390fb6ab51f2/retrieval_head_detection.py#L188 For "llama-2-7b-80k" model, why we need to reset the rope parameters? It seems that the config.json have included the scale_factor.

cxiliao

I would like to reopen the issue: https://github.com/nightdessert/Retrieval_Head/issues/6#issuecomment-3183153245 Any help from the authors is highly appreciated. Thanks very much!

nlokeshiisc

It's strange that the colon didn't cause an error, and the code continued to execute, but it didn't take effect .

relic-yuexi