Elwood

icon indicating a website link https://chezacar.github.io icon indicating an email [email protected]

icon company Shanghai Jiao Tong University icon location Shanghai icon location Wind of Nanyang

Results 2 comments of Elwood

【bug】 multi-note grpo block, single node is ok

I also suffer this, can you give a complete example, please?

Running the GRPO program on multiple nodes causes it to hang.

I encountered the same issue where the FSDP parameter offloading appears to be causing the process to hang. Interestingly, even after disabling parameter offloading (setting offload_params=False), the process still gets...