llama-stack
llama-stack copied to clipboard
" errors" when processing output stream
Every so often I'll get something like this in my output stream when doing async processing
{"event":{"event_type":"progress","delta":" errors","logprobs":null,"stop_reason":null}}
here's the code reading and printing the response, it's a modified client:
# Process the response from the server
async for log in client.chat_completion(request):
# print(f"Log structure: {log}\n{type(log)}\n")
# print(f"\nLog structure: {log}\n{type(log)}\n{log.event}\n{type(log.event)}\n")
if isinstance(log, ChatCompletionResponseStreamChunk):
# print(f"RESPONSE STREAM CHUNK")
if isinstance(log.event, ChatCompletionResponseEvent):
# print("\tRESPONSE EVENT")
if isinstance(log.event.event_type, ChatCompletionResponseEventType):
if log.event.event_type == ChatCompletionResponseEventType.start:
cprint(f"\n", "white", flush=True)
elif log.event.event_type == ChatCompletionResponseEventType.complete:
_stop_reason = log.event.stop_reason
cprint(f"\nCOMPLETED::Stop Reason::[{_stop_reason}]\n", "red", flush=True)
break
elif log.event.event_type == ChatCompletionResponseEventType.progress:
if ("errors" in log.event.delta and log.event.logprobs == None and log.event.stop_reason == None):
break
else:
cprint(log.event.delta, "cyan", end='', flush=True)
else:
print(f"""
Log structure: {log}
{type(log)}
{log.event}
{type(log.event)}
---------------------------------------------------------------------
print(f"\t\tEVENT TYPE ::[{log.event.event_type}]")
print(f"\t\tEVENT [type] TYPE ::[{type(log.event.event_type)}]")
print(f"\t\tEVENT TYPE DELTA ::[{log.event.delta}]")
print(f"\t\tEVENT TYPE LOGPROBS ::[{log.event.logprobs}]")
print(f"\t\tEVENT TYPE STOP_REASON ::[{log.event.stop_reason}]")
---------------------------------------------------------------------
UNHANDLED ChatCompletionResponseEventType::[{log.event.event_type}]
""")
why do I get the errors every so often and my attempts to catch them and skip seems to fail.
Any suggestions?