Jonathan Bratt
Jonathan Bratt
currently, we test with BERT_base, but we may as well use the smallest available.
It doesn't complain right away, but if you run enough models, you get a message like: > WARNING:tensorflow:5 out of the last 6 calls to triggered tf.function retracing. Tracing is...
currently, I attach tt_ids as an attribute to the tokenized input in `tokenize_input`. It feels like a misuse of attributes, but I also don't want to, say, pass the tt_ids...
I think we decided this would be a good thing to do.
The filename says it all. These were hastily written to get the branch into a working state, and should be refactored with more attention paid to speed and safety.
There are several tokenization conventions (e.g. the token used for padding, separating segments, etc.) that need to be specified when doing the wordpiece tokenization for BERT. Currently, some of these...
`run_classifier.R` is mostly about fine-tuning BERT with a classifier head. RBERT is not quite working at this level yet. There are almost certainly bugs that would prevent this from working...
The original BERT checkpoints released by Google are in a TensorFlow format. It seems that most of the related work done by other teams is in the [PyTorch](https://github.com/huggingface/pytorch-transformers) implementation. In...