Richard Li
Richard Li
We'd love to have a community member contribute a PR on this! @asitaru? @prakash-halan?
@matthyx The python script would be great! (or better yet, would you be interested in a guest blog post?)
@matthyx Yes, we're planning on revamping the tutorial section on the Telepresence website, so it could fit there. And yes, of course you could publish it on your blog! (Happy...
@robertgshaw2-neuralmagic Note that a previous PR for this functionality was rejected, see https://github.com/vllm-project/vllm/pull/1244. I'm not sure if the goal of `api_server` is just "demo ware you should fork", as suggested...
Thanks for the detailed feedback! Definitely something that makes sense and we need to work towards this in future releases.