Streaming chat responses broke on 0.0.63

[roundpanda]

The bug After update to 0.0.63 streaming chat responses from OpenAI broke. On previous version i received data as it came from OpenAI now it waits till the request is completly finished to return all data at once.

To Reproduce

    guidance.llm = guidance.llms.OpenAI("gpt-3.5-turbo")
    program = guidance('''
    {{#system~}}
    You are helpful assistant.
    {{~/system}}
    {{#user}}
    Write me a long poem.
    {{/user}}
    {{#assistant~}}
        {{gen "answer" save_prompt="prompt" max_tokens=300}}
    {{~/assistant}}
    ''') # type: ignore

    pos = 0
    for p in program(stream=True, silent=True, caching=False):
        val = p.get("answer", "")
        print(val[pos:], end="", flush=True)
        pos = len(val)

System info:

• Guidance Version (guidance.__version__ = "0.0.63"):

[HalfMAI]

I tested it and found that version 0.061 can streaming generate text, while the issue only appears after version 0.062.

[marcotcr]

Hopefully it's working in the new release! Please let us know if not :)