A1waysBeenHere

icon company UIUC icon location Hangzhou icon location Gameboy From University of Illinois Urbana-Champaign

Results 1 comments of A1waysBeenHere

Out of Memory running Qwen3-30B-A3B with 32k sequence length on 4 nodes (32 GPUs)

Seems the params cannot be offloaded correctlly. What's the global batch size you used btw?