Openai triton server

[nnshah1]

[dongs0104]

hello i have question about triton server running using python code import tritonserver

i have 4 gpus for serve, so i have to set world size 4 but i can not found any option or tutorials, if you can help I'd be so grateful.

[thehumit]

hello i have question about triton server running using python code import tritonserver

i have 4 gpus for serve, so i have to set world size 4 but i can not found any option or tutorials, if you can help I'd be so grateful.

I've bumped into same issue, hav you find solution?

[rmccorm4]

Please see this new location for details on an OpenAI-compatible Frontend for Triton: https://github.com/triton-inference-server/server/tree/main/python/openai