韩森宇 issues

Repositories
Issues
Comments

Results 1 issues of


                                            韩森宇

Question about blocking an attention head

https://github.com/nightdessert/Retrieval_Head/blob/3ac171a6f71ce7ef1cda57d4215c390fb6ab51f2/faiss_attn/source/modeling_llama.py#L685-L689 Hi there, I'm working on the interpretability of attention heads and your work is really inspiring. I'm not pro in modifications on these attention heads, so I'm a little...