Dimadon comments

Repositories
Issues
Comments

Results 4 comments of


                                            Dimadon

feat: OpenAI Compatible Frontend

Hello, is it already clear when the beta phase will be over? Thanks for this very important PR!

feat: OpenAI Compatible Frontend

Hi, is there a timeline for this PR? This would be such a great feature 🤩

feat: OpenAI Compatible Frontend

Hi @oandreeva-nv, Thanks. I have tested this feature. Did you know when its leaves the beta status?

Problems with the response of the OpenAI-Compatible Frontend for Triton Inference Server

Hi @rmccorm4, Thanks for the explanation. That makes sense. Is it possible to override the default in vllm. I have an application where I can not set the max_token. Thanks...