ValueError: `rope_scaling`'s type field must be one of ['su', 'yarn'], got longrope

[jzyee]

attempted: https://github.com/Leeroo-AI/mergoo/blob/main/notebooks/integrate_phi3_experts.ipynb

when loading the model with:

import torch from mergoo.models.modeling_phi3 import Phi3ForCausalLM

model_id = 'data/checkpoint_demo' // Define the device (use cuda:0 or another GPU if necessary) device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu') model = Phi3ForCausalLM.from_pretrained( model_id, device_map=device, torch_dtype=torch.bfloat16 )

and got the following error: ValueError: rope_scaling's type field must be one of ['su', 'yarn'], got longrope

transformers==4.44.2 torch==2.0.1