[Proposal] Introducing an API Endpoint for Token Count and Cost Estimation

[StephenHodgson]

As the development community continues to leverage OpenAI’s advanced models, the need for more transparent and accessible budgeting tools has become apparent. The introduction of a programmatic way to estimate token usage and associated costs would mark a significant step forward in this regard, particularly with the utilization of cutting-edge models like GPT-4o.

This proposal outlines the introduction of a dedicated API endpoint for token count and cost estimation and suggests enhancing model objects with pricing information to aid developers in making more informed decisions.

The Proposal

1. API Endpoint for Token Count and Cost Estimation (count_tokens): This endpoint aims to provide developers with an efficient tool for estimating the number of tokens generated by their text inputs, alongside the expected cost, for a specific model, namely GPT-4o.
2. Incorporate Pricing Information into Model Endpoints: To further aid decision-making processes, it is proposed that model detail endpoints be updated to include essential pricing information.

Benefits

• Accurate Cost Management: Enables developers to accurately manage and forecast their expenditures on the OpenAI platform.
• Seamless Developer Workflow: Integrates directly into development workflows, allowing for real-time cost estimations without manual intervention.
• Transparent Pricing: Offers clear visibility into pricing structures, promoting trust and reliability in the OpenAI ecosystem.

Suggested Implementation

Here is how the count_tokens endpoint could be implemented, using GPT-4o as an example:

POST /v1/count_tokens
Content-Type: application/json

{
    "model": "gpt-4o",
    "text": "Your sample text goes here."
}

Expected Response:

{
    "tokens": 150,
    "cost": 0.001
}

To include model-specific pricing details transparently:

GET /v1/models

Sample Response:

{
    "models": [
        {
            "id": "gpt-4o",
            "object": "model",
            "token_cost_per_thousand": "0.08"
        },
        // More model objects...
    ]
}

The introduction of the count_tokens endpoint, particularly with support for GPT-4o, represents a critical enhancement to the OpenAI API, streamlining development processes and facilitating better budget management. Community feedback on this proposal is invaluable for refining and implementing these suggestions effectively.

[supersteves]

It perplexes me why OpenAI provide token counts in responses and have a html pricing page, but don't provide an endpoint to retrieve the same information (dollar pricing per 1M tokens, by model).

For a developer that wishes to provide real-world pricing information to end users, it requires hard-coding pricing data into the app, which will likely go out of date, and need updating for new models.