Transformers.jl
Transformers.jl copied to clipboard
DistilBertModel support
Hi there,
First of all, thanks for the wonderful package you have done.
I'm trying to load a HuggingFace Model but I got these warnings and an error:
using Transformers
using Transformers.TextEncoders
using Transformers.HuggingFace
textenc, model = hgf"sentence-transformers/distiluse-base-multilingual-cased-v1"
┌ Warning: startsym <s> not in vocabulary, this might cause problem.
└ @ Transformers.TextEncoders [/home/ajaimes/.julia/packages/Transformers/lD5nW/src/textencoders/bert_textencoder.jl:77](https://file+.vscode-resource.vscode-cdn.net/home/ajaimes/.julia/packages/Transformers/lD5nW/src/textencoders/bert_textencoder.jl:77)
┌ Warning: endsym </s> not in vocabulary, this might cause problem.
└ @ Transformers.TextEncoders [/home/ajaimes/.julia/packages/Transformers/lD5nW/src/textencoders/bert_textencoder.jl:78](https://file+.vscode-resource.vscode-cdn.net/home/ajaimes/.julia/packages/Transformers/lD5nW/src/textencoders/bert_textencoder.jl:78)
Unknown model type: distilbert
Do you have in mind adding support for this type of models (distilbert)? Or is there some way to achieve this?
Thank you in advance!