llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

" errors" when processing output stream

Open teamblubee opened this issue 1 year ago • 3 comments

Every so often I'll get something like this in my output stream when doing async processing

{"event":{"event_type":"progress","delta":" errors","logprobs":null,"stop_reason":null}}

here's the code reading and printing the response, it's a modified client:

        # Process the response from the server
        async for log in client.chat_completion(request):
            # print(f"Log structure: {log}\n{type(log)}\n")
            # print(f"\nLog structure: {log}\n{type(log)}\n{log.event}\n{type(log.event)}\n")
            if isinstance(log, ChatCompletionResponseStreamChunk):
                # print(f"RESPONSE STREAM CHUNK")
                if isinstance(log.event, ChatCompletionResponseEvent):
                    # print("\tRESPONSE EVENT")
                    if isinstance(log.event.event_type, ChatCompletionResponseEventType):
                        if log.event.event_type == ChatCompletionResponseEventType.start:
                            cprint(f"\n", "white", flush=True)
                        elif log.event.event_type == ChatCompletionResponseEventType.complete:
                            _stop_reason = log.event.stop_reason
                            cprint(f"\nCOMPLETED::Stop Reason::[{_stop_reason}]\n", "red", flush=True)
                            break
                        elif log.event.event_type == ChatCompletionResponseEventType.progress:
                            if ("errors" in log.event.delta and  log.event.logprobs == None and log.event.stop_reason == None):
                                break
                            else:
                                cprint(log.event.delta, "cyan", end='', flush=True)
                        else:
                            print(f"""
                            Log structure: {log}
                            {type(log)}
                            {log.event}
                            {type(log.event)}
                            ---------------------------------------------------------------------
                            print(f"\t\tEVENT TYPE             ::[{log.event.event_type}]")
                            print(f"\t\tEVENT [type] TYPE      ::[{type(log.event.event_type)}]")
                            print(f"\t\tEVENT TYPE DELTA       ::[{log.event.delta}]")
                            print(f"\t\tEVENT TYPE LOGPROBS    ::[{log.event.logprobs}]")
                            print(f"\t\tEVENT TYPE STOP_REASON ::[{log.event.stop_reason}]")
                            ---------------------------------------------------------------------
                            UNHANDLED ChatCompletionResponseEventType::[{log.event.event_type}]
                                """)

why do I get the errors every so often and my attempts to catch them and skip seems to fail.

Any suggestions?

teamblubee avatar Sep 12 '24 10:09 teamblubee