Mergekit support for GPT2 failing

[NamburiSrinath]

Hi @cg123,

Great library, thanks a lot, super useful!

I've finetuned GPT2 on 2 tasks (model1 and model2) and am trying to merge using your repo.

It turns out, using the original model as the base throws one or the other error (more details here - https://github.com/huggingface/transformers/issues/29184 as I've raised it)

But using one of the model as the base model (base_model: model1) is working fine. Is this expected?

Thanks a ton, looking forward.