OpenFactVerification icon indicating copy to clipboard operation
OpenFactVerification copied to clipboard

Add token count to return of FactCheck().check_response

Open evangriffiths opened this issue 10 months ago • 1 comments

It would be nice to be able to track tokens used by a fact check call using the python api, e.g. something like:

from factcheck import FactCheck
results = FactCheck().check_response("The sky is green")
print(results["token_count"])

which would contain prompt and completion token info, like

{'num_raw_tokens': 4, 'num_checkworthy_tokens': 5, 'total_prompt_tokens': 1748, 'total_completion_tokens': 231}

Hacky example implementation for openai client only here: https://github.com/Libr-AI/OpenFactVerification/pull/11

evangriffiths avatar Apr 19 '24 10:04 evangriffiths