DNABERT
DNABERT copied to clipboard
Add custom tokens to DNATokenizer
Hi, I am wondering is it possible to add tokens to DNATokenizer?
As the function tokenizer = AutoTokenizer.from_pretrained('bert-base-cased') tokenizer.train_new_from_iterator(owndata) which can add own tokens to bertTokenizer.