agents icon indicating copy to clipboard operation
agents copied to clipboard

Add token_usage_callback to openai llm plugin

Open willsmanley opened this issue 6 months ago • 1 comments

Now consumers can receive token usage metrics after each LLM completion event with the openai LLM plugin:

    def token_usage_callback(token_usage: dict[str, int]) -> None:
        cost_per_prompt_token = 2.5 / 1000000 # see: https://openai.com/api/pricing/
        cost_per_completion_token = 10 / 1000000
        money_down_the_tube = token_usage.prompt_tokens * cost_per_prompt_token + token_usage.completion_tokens * cost_per_completion_token
        print(money_down_the_tube)
    
    llm = openai.LLM(
        model='gpt-4o-2024-08-06',
        token_usage_callback=token_usage_callback,
    )

See Issue: https://github.com/livekit/agents/issues/613

willsmanley avatar Aug 11 '24 01:08 willsmanley