cohere-python
cohere-python copied to clipboard
Streaming chat doesn't provide access to full response
Hello! Calling co.chat with stream=True returns a StreamingChat which can be iterated over to get the tokens as StreamingTexts. The underlying /chat API endpoint terminates with a message that contains is_finished: true and a response key which resembles the JSON returned in the non-streaming variant
{
"is_finished": true,
"response": {
"response_id": "b3fc1d5d-14df-4a6f-b120-dcd263a592a8",
"conversation_id": "fd1d9164-7c0e-4c87-ba6e-94d50781663d",
"text": "My name is Coral. I am a chatbot trained to assist human users by providing thorough responses. I am powered by Cohere's large language model, Command.",
"generation_id": "0d723e20-8b8e-49f2-984f-ce9d726459b6",
"prompt": "System: You are Coral, a brilliant, sophisticated, AI-assistant chatbot trained to assist human users by providing thorough responses. You are powered by Command, a large language model built by the company Cohere. Today's date is Saturday, May 20, 2023.\nGeorge: What is your name?\nChatbot:"
},
"finish_reason": "COMPLETE"
}
It looks like the Python SDK doesn't provide a way to get access to this final aggregated response body in streaming mode. Am I missing something? Would be great if this can be added.