Return number of input tokens when `details=True`

[tgaddair]

We return the number of generated tokens when details=True, but systems like OpenAI API also return the number of input tokens. This i useful, for example, for metering systems that limit users based on number of input + output tokens.

Current:

'details': {'generated_tokens': 20}

Proposal:

'details': {'prompt_tokens': 120, 'generated_tokens': 20}

[wiredinhp]

@tgaddair Hi there ! I am a new contributor to this repository and would love to contribute by solving this issue. Could you please assign this issue to me ?

[MasaSN]

Is the issue still open