EdgeChains
EdgeChains copied to clipboard
JS - add Claude BPE Tokenizer
https://twitter.com/AlexReibman/status/1704212821607645332 https://github.com/anthropics/anthropic-tokenizer-typescript
https://github.com/anthropics/anthropic-tokenizer-typescript/blob/bd241051066ea37120f2898419e3fc8662fab280/index.ts#L3C1-L3C1