openai-streaming
openai-streaming copied to clipboard
AlmogBaku
Work with OpenAI's streaming API at ease with Python generators
Consider using pydantic allow partial instead of json streaming
https://github.com/AlmogBaku/openai-streaming/blob/master/openai_streaming/struct/handler.py#L95-L96 This should not initiate validation unless its the final response
Instead of returning only the “functions called” and “unstreamed response message”, return a comprehensive bag of information as a Response class that contains: 1. Functions called 2. Unstreamed reaponse messages...
>Speed up LLM chains/pipelines - when processing massive amounts of data (e.g., classification, NLP, data extraction, etc.), every bit of speed improvement can accelerate the processing time of the whole...