backstage-plugin-toolbox icon indicating copy to clipboard operation
backstage-plugin-toolbox copied to clipboard

Published 20 hours ago • verified publisher icon drodil

Feature: Tokenizer

Open billabongrob opened this issue 9 months ago • 0 comments

It would be really cool to have a Tokenizer similar to https://www.npmjs.com/package/tiktoken available to approximate token counts on text in order to estimate costs in certain tools.

The nice part about this would be that it keeps the data internal to your organization so you don't have to post into external tools.

billabongrob avatar May 17 '24 14:05 billabongrob