Token Usage tracking doesn't seem to work for ChatGPT (with streaming off)

[bent-verbiage]

Using the callback method described here, the Token Usage tracking doesn't seem to work for ChatOpenAI. Returns "0" tokens. I've made sure I had streaming off, as I think I read that the combination is not supported at the moment.

https://langchain.readthedocs.io/en/latest/modules/llms/examples/token_usage_tracking.html?highlight=token%20count

[stephenleo]

I can work on this

[stephenleo]

just submitted a PR: https://github.com/hwchase17/langchain/pull/1748

[stephenleo]

submitted another PR: #1924 Would love to get some feedback as this is blocking for me :(

[bent-verbiage]

Thanks for you work on this @stephenleo. I saw that @hwchase17 wants to fix this in a different way, but your work is still appreciated!

[dosubot[bot]]

Hi, @bent-verbiage! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the Token Usage tracking feature in ChatGPT is not functioning correctly when streaming is disabled, and it returns "0" tokens. @stephenleo has offered to work on this and has submitted two pull requests (#1748 and #1924). You thanked @stephenleo for their work, and it seems that @hwchase17 wants to fix this in a different way.

Before we proceed, could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project! Let me know if you have any further questions or concerns.