Ze Ch

When running multi-gpu programs such as deepspeed, if gg bash is used, the program will hang. Is it possible that gg changed some config that makes multi-gpu comm impossible?

[Feature]: continuous batching for vllm.LLM

2

### 🚀 The feature, motivation and pitch Motivation: I'm using vllm.LLM to instantiate multiple vllm engines. During training, I want to make a large number of one-prompt requests to the...