Wang Binluo

icon company colossalai icon location Singapore icon location Previous worked in ByteDance, Temu. Master of NUS, bachelor of SYSU. Focus on parallel training in LLMs.

Results 8 issues of Wang Binluo

[npu] fix the build before load bug

## 📌 Checklist before creating the PR - [ ] I have created an issue for this PR for traceability - [ ] The title follows the standard format: `[doc/gemini/tensor/...]:...

bug

[shardformer] Support Qwen2 model

## 🚨 Issue number - [ ] https://github.com/hpcaitech/ColossalAI/issues/5573 ## 📝 What does this PR do? [shardformer/modeling/qwen2]: add qwen2.py and qwen2 policy to support qwen2 model, have passed all the tests...

[FEATURE]: Support qwen2 model

### Describe the feature We are excited to announce the addition of support for the qwen2 model in the ColossalAI framework. The qwen2 model is compatible with version 4.39.3 of...

enhancement

[FEATURE]: Upgrade the transformers version from 4.33.0 to 4.36.0 for Shardformer.

1 comment

### Describe the feature Shardformer was originally developed based on transformers==4.33.0. In response to our users' needs, it needs to be upgraded to version 4.36.0. The main changes involve the...

enhancement

[Shardformer] Add Parallel output for shardformer models

Support parallel output function for shardformer models.

[CUDA] FP8 all-reduce using all-to-all and all-gather

qwen2 fp8 forward/backward

[doc] sequence parallel document

## 📝 What does this PR do? Supplementary comparison of the principles of sequence parallel, including ring-attention and Ulysess, and an explanation of their use cases.