Support BART models for classification

[juliensalinas]

Hello again,

I'm trying to convert this adaptation of Bart Large MNLI: https://huggingface.co/joeddav/bart-large-mnli-yahoo-answers

It returns the following error (but the base Bart Large MNLI model works well):

Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/./ct2-transformers-converter", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.10/site-packages/ctranslate2/converters/transformers.py", line 445, in main
    converter.convert_from_args(args)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/ctranslate2/converters/converter.py", line 50, in convert_from_args
    return self.convert(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/ctranslate2/converters/converter.py", line 89, in convert
    model_spec = self._load()
  File "/home/ubuntu/.local/lib/python3.10/site-packages/ctranslate2/converters/transformers.py", line 62, in _load
    return loader(self._model_name_or_path)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/ctranslate2/converters/transformers.py", line 85, in __call__
    spec = self.get_model_spec(model)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/ctranslate2/converters/transformers.py", line 146, in get_model_spec
    pre_norm=model.config.normalize_before,
  File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 257, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'BartConfig' object has no attribute 'normalize_before'

Thanks in advance!

[guillaumekln]

Lots of different configuration to consider. :)

Thank you again for reporting.

[guillaumekln]

Actually this model is not fully supported. It uses the architecture BartForSequenceClassification, but we don't support the additional classification head at the moment.

[juliensalinas]

Thanks for investigating @guillaumekln !

[vince62s]

I am seconding this. It would be great to implement Bert-like models with encoders only + classification head. More specifically if we can use pre-trained parser like this: https://ufal.mff.cuni.cz/udpipe/2/models it would make things easier to integrate in pipelines.

[guillaumekln]

This issue is for BART, the sequence to sequence model. I created another issue for encoder only models like BERT: #1008.