langchain
langchain copied to clipboard
support deepseek's token usage format
Deepseek includes a choice with empty content in their token usage chunk when streaming. otherwise, their API is compatible with OpenAI's. This commit adds support for this chunk format.
Example of a token usage chunk from Deepseek:
data: {
"id":"339eade8-b499-472e-bc0d-b267b6e4bceb",
"object":"chat.completion.chunk",
"created":1743968719,
"model":"deepseek-chat",
"system_fingerprint":"fp_3d5141a69a_prod0225",
"choices":[
{
"index":0,
"delta":{
"content":""
},
"logprobs":null,
"finish_reason":"stop"
}
],
"usage":{
"prompt_tokens":11,
"completion_tokens":11,
"total_tokens":22,
"prompt_tokens_details":{
"cached_tokens":0
},
"prompt_cache_hit_tokens":0,
"prompt_cache_miss_tokens":11
}
}
@brainlid I think I added the test you've asked for. Makes total sense.
However, looking at the test, I'm not sure we should emit a MessageDelta when the content = "". Would it make more sense to return :skip in those cases?
Hi @manukall! Please check out the v0.4.0-rc.0 release. There are other breaking changes that I expect are needed for DeepSeek to be supported, but your TokenUsage update would be best applied to that branch as TokenUsage is being stored in the received Message's metadata.usage.