Fine-tuning a 13B mt0-xxl model

[QLutz]

Hello and thanks for the awesome library !

I'd like to reproduce some of the results you display in the repo's README and had a few questions:

1. I was wondering what you meant by

Also, we didn't use the larger 13B mt0-xxl model

as written in the Use cases section of the README ?

2. What code did you use for the PEFT-LoRA PyTorch x mt0-xxl datapoint? I'm assuming it's the one from examples/conditional_generation/peft_lora_seq2seq.ipynb but I run into OOM errors when I try on a similar setup (i.e. A100 80GB + fair amount of RAM).

Thanks in advance :)