Errors with Phi3 models and how to run the code with the model Yi-6B-200K?

[HayeonLee]

Hi, thank you for the awsome code!

When I run python retrieval_head_detection.py --model_path microsoft/Phi-3-mini-128k-instruct --s 0 --e 50000, I got the following errors:

Could you share the tips to fix this error or the transformer version you used?

Thank you!

[shaswatpatel123]

Hey @HayeonLee , I faced the similar issue. I solved it by using the following code: https://huggingface.co/microsoft/Phi-3.5-mini-instruct/blob/main/modeling_phi3.py

You need to add Phi3LongRoPEScaledRotaryEmbedding class from the above code into modeling_phi3.py file. Also add type longrope here and call the Phi3LongRoPEScaledRotaryEmbedding class. You can check the above code for reference.