Hooman

Hi all, I think it will be a great success to extend this to `token-classification` too. Have you thought about something along the lines of the followings: - Keybert (https://github.com/MaartenGr/KeyBERT)...

@krrishdholakia this issue still occurs occasionally. In my case with `"model": "gpt-35-turbo-1106"` and I am using `1.19.2` ```python 6741, in exception_type raise e File "python3.11/site-packages/litellm/utils.py", line 6680, in exception_type raise...

Hi @willydouhard this is a quite nice solution. Just please note whether ‘streaming’ will work too in this scenarios. I see that you have created a callback handler for chainlit...

@willydouhard I can also test tmrw morning once I’m next to my laptop ! But this is really amazing! Congrats again, this is going to bring chainlit to the commercial...

> If you're using FastAPI, the author (Tiangolo) has a nice project called Asyncer, which has a very nice [asyncify function](https://asyncer.tiangolo.com/tutorial/asyncify/). This is just a wrapper on top of anyio...

Thanks a lot @ewellinger . Have you tried to benchmark your approach against the following using `asyncify` ? ```python app = FastAPI() @app.get("/completion") async def get_completion(): return await get_huggingface_completion("Please give...

Awesome @ewellinger , thanks a lot! looks like your approach is slightly better. SageMaker has recently introduced [`invoke_endpoint_with_response_stream`](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime/client/invoke_endpoint_with_response_stream.html) but still is a synchronous operation (in sagemaker python sdk) but with...