DNABERT icon indicating copy to clipboard operation
DNABERT copied to clipboard

Add custom tokens to DNATokenizer

Open WENHUAN22 opened this issue 2 years ago • 0 comments

Hi, I am wondering is it possible to add tokens to DNATokenizer?

As the function tokenizer = AutoTokenizer.from_pretrained('bert-base-cased') tokenizer.train_new_from_iterator(owndata) which can add own tokens to bertTokenizer.

WENHUAN22 avatar Jul 07 '22 12:07 WENHUAN22