importing checkpoints to transformer library

[eghbalhosseini]

I am trying to import the weights from one of the models I pre-trained using gpt2-neox in to transformer library for some downstream tests. I used AutoModel.from_pretrained(path to checkpoint,model_config) However it cannot load the model weights appropriately. is there a solution to this, or what I am doing wrong here?

[eghbalhosseini]

By the way, I am trying to write one based on the following :

https://github.com/huggingface/transformers/blob/2e4559fa375225ebe7d2c45fa96eccca794068b2/src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py#L42