agents
agents copied to clipboard
Add token_usage_callback to openai llm plugin
Now consumers can receive token usage metrics after each LLM completion event with the openai LLM plugin:
def token_usage_callback(token_usage: dict[str, int]) -> None:
cost_per_prompt_token = 2.5 / 1000000 # see: https://openai.com/api/pricing/
cost_per_completion_token = 10 / 1000000
money_down_the_tube = token_usage.prompt_tokens * cost_per_prompt_token + token_usage.completion_tokens * cost_per_completion_token
print(money_down_the_tube)
llm = openai.LLM(
model='gpt-4o-2024-08-06',
token_usage_callback=token_usage_callback,
)
See Issue: https://github.com/livekit/agents/issues/613