revtok
revtok copied to clipboard
Published
20 hours ago •
jekbradbury
jekbradbury
Reversible tokenization in Python.
Results
1
revtok issues
Sort by
recently updated
recently updated
newest added
Hey, when calling this module from torchtext, the default `max_size`is None, which gets propagated to SubwordSegmenter and causes a not-so obvious error (in the tqdm loop, or even more obfuscated...
