fabiocapsouza

Hi domus123, The models were trained on TPU v3-8 using TF 1.14 and the code from original BERT repository: [https://github.com/google-research/bert/blob/master/run_pretraining.py](https://github.com/google-research/bert/blob/master/run_pretraining.py). Unfortunately I'm not a TF user so I can really...

Hi @lkqnaruto , By interleaved zeros and ones, I meant a mask like `[0, 1, 0, 1, 1, 0, 0, 0, 1, ...]` instead of `[1, 1, 1, 1, 0,...

Hi @ViktorooReps , I used the first subtoken because it is the way BERT does it for NER, so it is the simplest way to add CRF on top of...

Hi @monilouise, Unfortunately, we did not implement ROC AUC because it is not used by the evaluation dataset we used, but it would be an interesting metric to have. Regarding...

Hi @elymatos, I don't use spaCy so I can't help you unfortunately. But if spaCy supports BERT models of the Transformers library, it should work with BERTimbau, since the only...

Hi @lkqnaruto , Yes, you can do that if you want to initialize the CRF layer with such constraints, but the pytorch-crf library does not expose an API for that,...

Yes, I believe so. I see it as a form of model initialization similar to [adjusting the bias terms of a classification layer to produce the prior probabilities of the...

Hi @Benjamim-EP , I am not a TensorFlow user, so unfortunately I can't give you directions. But it should be possible to adapt a working example for English BERT (or...

Hi @Phd-Student2018 , I don't know if I understood your question, but here is an example of this indexing: suppose we have the following words, tokens and labels ```python words...

Hi @monilouise, which version of Pytorch are you using? If you are not using 1.1.0 (version the code was written for), the function `pack_padded_sequence` inputs were changed to accept CPU...