bumblebee icon indicating copy to clipboard operation
bumblebee copied to clipboard

Published 20 hours ago • verified publisher icon elixir-nx

Add CamemBERT

Open seanmor5 opened this issue 2 years ago • 1 comments
trafficstars

This was requested specifically for NER tasks. It's actually just a Roberta model, so there's no new implementation, but the config lists CamemBERT so it throws off our loader.

@jonatanklosko Let me know if you think it is worth adding an extra module of integration tests to make sure all of the mappings load correctly :)

AFAICT there is not a fast tokenizer, but I think users can get around this manually by loading in Python and using convert_slow_to_fast or whatever the conversion is in Python

seanmor5 avatar Dec 14 '22 00:12 seanmor5

@seanmor5 maybe a single test for camembert-base in the roberta suite? I don't think we need to test all mappings.

As for the tokenizer, there is tokenizer.json in camembert-base, so we should be able to add it?

jonatanklosko avatar Dec 14 '22 00:12 jonatanklosko