Wang Binluo

icon company colossalai icon location Singapore icon location Previous worked in ByteDance, Temu. Master of NUS, bachelor of SYSU. Focus on parallel training in LLMs.

Results 3 comments of Wang Binluo

[BUG]: OOM during llama2 pretraining with flashattention and PP

> @wangbluo Could you please help me solve this issue? Thanks Hi, could you please offer the model size you use?

[BUG]:

@lyzKF Hello, I have the same error with you, when I tried to run: colossalai run --nproc_per_node 8 --host 10.90.5.14,10.90.8.153 --master_addr 10.90.5.14 auto_parallel_with_gpt.py I got the error : Error: failed...

[FEATURE]: Upgrade the transformers version from 4.33.0 to 4.36.0 for Shardformer.

> Hi! Thanks for this work. I am not a part of hpcaitech, but just wondering: are you planning to keep bumping the version up? Since HF transformers version is...