Michael Kotlikov

Results 1 comments of Michael Kotlikov

A good use cade for a `Decode()` method would be a `TokenTextSplitter()` method. It seems to be reversible because other libraries have working decode methods such as: https://github.com/hyunwoongko/gpt2-tokenizer-java/blob/master/src/main/java/ai/tunib/tokenizer/GPT2Tokenizer.java.