crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

Expose completion tokens, total tokens, cost, etc. on OpenAI

Open DrakkoFire opened this issue 1 year ago • 1 comments
trafficstars

A good feature IMO is to have the tokens used exposed, in order to have a good idea on how many tokens we used on each request when storing analytics of our workers / APIs, in order to know how much did each query cost, and can know how much each user costs us.

I would gladly open a PR if needed with that update if more people find this useful as I do, as not only the response would be good to have, but the cost of each completion.

Crawl4AI uses LiteLLM under the hood, which themselves expose these parameters on their responses under the following key: completion_cost (See reference here), so it would just be a matter of scaling that info to the response given by Crawl4AI from ./crawl4ai/extraction_strategy.py:116 to the final user who's using this package in order to do whatever they want with it.

DrakkoFire avatar Oct 27 '24 18:10 DrakkoFire

@DrakkoFire thank you for the suggestion, I believe you're right. Definitely, in the new version, I'll add the data I get back from the Light LLM as a response, so you can explore the token usage as well.

unclecode avatar Nov 04 '24 07:11 unclecode