tokenizers
tokenizers copied to clipboard
Any plan to binding to java?
I was curious about this, too. I might try to cobble together a PR to generate JNI bindings.
We'd welcome a PR.
There was a very early effort by @mfuntowicz on https://github.com/huggingface/tokenizers/tree/java-binding using Kotlin, maybe it can be a good starting point (or feel free to start from scratch again if you prefer!)