Ispanicus

Timeout Error in all_gather during evaluate_ppo() on 2 H100 GPUs with miniLLM and ZeRO

2

Hi, I'm using ZeRO with optimizer and parameter offload to run minillm on 2 H100 gpus on a single node. After doing the generation evaluation, I get a timeout during...