cohere-python icon indicating copy to clipboard operation
cohere-python copied to clipboard

Streaming chat doesn't provide access to full response

Open seabo opened this issue 2 years ago • 0 comments

Hello! Calling co.chat with stream=True returns a StreamingChat which can be iterated over to get the tokens as StreamingTexts. The underlying /chat API endpoint terminates with a message that contains is_finished: true and a response key which resembles the JSON returned in the non-streaming variant

{
    "is_finished": true,
    "response": {
        "response_id": "b3fc1d5d-14df-4a6f-b120-dcd263a592a8",
        "conversation_id": "fd1d9164-7c0e-4c87-ba6e-94d50781663d",
        "text": "My name is Coral. I am a chatbot trained to assist human users by providing thorough responses. I am powered by Cohere's large language model, Command.",
        "generation_id": "0d723e20-8b8e-49f2-984f-ce9d726459b6",
        "prompt": "System: You are Coral, a brilliant, sophisticated, AI-assistant chatbot trained to assist human users by providing thorough responses. You are powered by Command, a large language model built by the company Cohere. Today's date is Saturday, May 20, 2023.\nGeorge: What is your name?\nChatbot:"
    },
    "finish_reason": "COMPLETE"
}

It looks like the Python SDK doesn't provide a way to get access to this final aggregated response body in streaming mode. Am I missing something? Would be great if this can be added.

seabo avatar May 20 '23 18:05 seabo