RoBERTa model not loadable

[scheiblr]

🐛 Bug

I just trained a RoBERTa GottBERT model and obtain the following error while trying to load the final checkpoint

omegaconf.errors.ConfigKeyError: Key 'include_index' not in 'MaskedLMConfig'
        full_key: include_index
        reference_type=Optional[MaskedLMConfig]
        object_type=MaskedLMConfig

To Reproduce

Steps to reproduce the behavior (always include the command you ran):

train some GottBERT model using fairseq-train and then load the checkpoint

gottbert = GottbertModel.from_pretrained('path/to/checkpoints', checkpoint_file='checkpoint_best.pt')

Expected behavior

Should load the model as usual

Environment

• fairseq Version (e.g., 1.0 or main): 0.12.2
• PyTorch Version (e.g., 1.0): 2.0.1
• OS (e.g., Linux): Ubuntu 20.04
• How you installed fairseq (pip, source): pip
• Build command you used (if compiling from source): pip install .
• Python version: 3.9.16
• CUDA/cuDNN version: -
• GPU models and configuration: -
• Any other relevant information: -

Additional context

with the installation from source the entire RoBERTa stuff seems to be broke. There I obrain allready errors by simply importing classes, such as

from fairseq.models.roberta import RobertaModel
# or
from fairseq.models.roberta import GottbertModel

[scheiblr]

Solved it by a workaround. I added the missing field to the checkpoint file;

import torch
model = torch.load("checkpoints/checkpoint.pt")
model['cfg']['include_index'] = False

torch.save(new_model, 'checkpoints/checkpoint_fixed.pt')

I'm not that sure, whether include_index should be false or true? I would be happy, if someone could give some feedback concerning that.