[Feature] Return token usage

[kieranx]

In the official OpenAI API, token usage is returned, e.g. "total_tokens": 1340

There seems to be no way to get this same info in LLPhant -- this is very basic stuff and I feel like it should be able to be accessed, given OpenAI do return that info? We can't figure out a way to do that in LLPhant currently however.

Edit: to be clear, this is for the questionanswering class, I know we can get it for the general chat class.

[f-lombardo]

What about https://github.com/theodo-group/LLPhant/blob/fea142b4e5b14095ae273166809abf77f0d68f9a/src/Chat/OpenAIChat.php#L74 ? Does it fit your needs?

[Vardan9898]

Is there a way to retrieve the total number of tokens used with Question Answering?

$qa = new QuestionAnswering( $memoryVectorStore, $embeddingGenerator, new OpenAIChat() );

$answer = $qa->answerQuestion('What is the secret of Alice?');

If not, is it possible to use OpenAIChat with vector-stored embeddings?

[f-lombardo]

Maybe you can try this:

    $chat = new OpenAIChat(new OpenAIConfig());
    $qa = new QuestionAnswering(
        $filesVectorStore,
        $embeddingGenerator,
        $chat
    );

    $response = $qa->answerQuestion('What is the secret of Alice?');

    echo('Total tokens: ' .$chat->getTotalTokens());

[MaximeThoonsen]

You want the total of token for the question embeddings + all generations?

[MaximeThoonsen]

I started a small PR here. We need to add the count of embeddings to make it correct