TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

Question about Orchestrator mode

Open akhoroshev opened this issue 9 months ago • 3 comments

Executor api introduces Leader and Orchestrator modes.

Leader works via mpi. How Orchestrator mode is implemented? Does it uses mpi itself? Which mode is preferable for performance: Leader or Orchestrator?

akhoroshev avatar May 15 '24 10:05 akhoroshev