griptape icon indicating copy to clipboard operation
griptape copied to clipboard

Can we add a way to estimate token size for Ollama?

Open shhlife opened this issue 5 months ago • 0 comments

Currently we have a TokenCounter utility that will count tokens for drivers other than Ollama. Ollama currently doesn't provide token information in it's responses: https://github.com/ollama/ollama/issues/1716, but we could perhaps use a tokenizer to estimate token size?

This might be useful as well for customers to estimate a call before they run it.

shhlife avatar Sep 06 '24 16:09 shhlife