DeepPavlov icon indicating copy to clipboard operation
DeepPavlov copied to clipboard

Published 20 hours ago • verified publisher icon deeppavlov

add roberta support

Open BunnyNoBugs opened this issue 3 years ago • 1 comments

Add RoBERTa support in squad_torch_bert config.

The changes concern the shape of token_type_ids specific for RoBERTa

BunnyNoBugs avatar Nov 11 '21 10:11 BunnyNoBugs

Now distilbert, roberta, xlm, bart and longformer-like models can be trained correctly (confirmed on a GPU)

BunnyNoBugs avatar Nov 25 '21 20:11 BunnyNoBugs