Tiktokenizer

Online playground for openai/tiktoken, calculating the correct number of tokens for a given prompt.

Special thanks to Diagram for sponsorship and guidance.

https://user-images.githubusercontent.com/1443449/222598119-0a5a536e-6785-44ad-ba28-e26e04f15163.mp4

Acknowledgments

• T3 Stack
• shadcn/ui
• openai/tiktoken