Marut Pandya
Marut Pandya
@avacaondata Sorry for this inconvenience. I can understand the frustration. We changed the flow a bit, for making it easy to update the vllm version but there's a model caching...
@ashwinb @yanxi0830 @hardikjshah when can I expect review? Thanks.
@ashwinb Sure. Thanks for the update. Looking forward to getting this merged soon.
@ashwinb let me update the implementation. Thanks
@ashwinb Closed the stale PR and Created a new one. Hoping this to be reviewed soon.
@ashwinb What does it require to have this PR reviewed ? Because, Stale PRs end up having merge conflicts and then it just becomes the another reason to not review/merge....
@raghotham Really appreciate you reviews. Totally understand the priorities and timeline. Hoping to getting this merged soon.
@michaelinva Do you mean in OpenAI completion API? We set it via ENV because it can be set during engine initialisation,
- Make sense. Let me work on that. Thanks for the feedback. @michaelinva
@casper-hansen Do you encounter this issue when using OpoenAI compat API?