tokenizations
tokenizations copied to clipboard
Support Python3.10
Thank you @tamuhey for supporting python 3.10. Is this draft still a work in progress? I'm looking forward to next release at https://pypi.org/project/pytokenizations/ with python 3.10. Or shall we switch to spacy-alignments
?
Hi @KoichiYasuoka, yes, please switch to spacy-alignments
instead of pytokenizations
. The most recent release v0.9.1 supports python 3.7-3.12.