tensorrtllm_backend
tensorrtllm_backend copied to clipboard
[Bugfix] Launch Triton server without waiting for a signal
Hi,
Problem:
This PR fix a silent bug inside the scripts\launch_triton_server.py module, this issue only occurs if we try to automatically launch the triton server inside a container using either CMD in the Dockerfile, or command in the docker-compose.yaml file.
For example in a Dockerfile:
CMD ["python3", "scripts\launch_triton_server.py", "--model_repo", "/workspace/model_repos/llama3_ifb", "--world_size", "1"]
Cause:
The cause of the issue is we're not wait for the signal when we call subprocess.Popen(cmd, env=env)
Before the change:
subprocess.Popen(cmd, env=env)
After the change:
# Start the subprocess and wait for signal
with subprocess.Popen(cmd, env=env) as proc:
try:
retcode = proc.wait()
except KeyboardInterrupt:
proc.kill()
return 0