griptape icon indicating copy to clipboard operation
griptape copied to clipboard

Prompt Events Redundantly Count Tokens

Open collindutter opened this issue 1 year ago • 0 comments

We calculate token_count as part of StartPromptEvent and FinishPromptEvent Events here.

This can be improved in the following ways:

  • For StartPromptEvent, we should calculate this value once in the BasePromptDriver and then pass the value to subclasses.
  • For FinishPromptEvent, many of the LLMs will return the token count in the response. We should not calculate it ourselves unless the value is missing.

collindutter avatar Dec 05 '23 18:12 collindutter