YeAnbang

Results 8 comments of YeAnbang

[ColossalChat] Update RLHF V2

Is ready for review now. Still working on fixing the following bug: - All script hang during booster.save_model with Gemini.

[ColossalChat] Update RLHF V2

# Local CI Test Results [ci.pdf](https://github.com/hpcaitech/ColossalAI/files/14138681/ci.pdf)

[ColossalChat] Update RLHF V2

all tests passed locally.

[ColossalChat] Update RLHF V2

# Experiment Report ## SFT ## DPO ![img_v3_029a_1201f32f-cdba-40e7-923e-d8ecc0f2d66g](https://github.com/hpcaitech/ColossalAI/assets/44796419/3869e10d-49ac-47cc-ac7b-5ee00394ef8c) ## PPO ![img_v3_029a_a70dba2a-7d80-4ee1-935f-67f5a47d17fg](https://github.com/hpcaitech/ColossalAI/assets/44796419/0239c550-798b-4978-b554-cf88c0fd728b) ![5FrWy9zwTr](https://github.com/hpcaitech/ColossalAI/assets/44796419/89c37e5d-5f4f-431a-bfca-bb9cf7aa6445)

[ColossalChat][WIP] Refactor coati; Fix PPO, LORA; Add DPO

Not ready for review. Thanks

[BUG]: `LowLevelZeroOptimizer` gets stuck in `all_reduce` when executing `check_overflow`

We don't support torch>=2.0. And since you are using CUDA 12.1 and pytorch 2.1.0, my suggestion will be downgrade your CUDA driver and pytorch.

[BUG]: ColossalChat train sft is skipped with opt-1.3b model

Hi, you can check if the dataset is empty by inserting "print(len(dataset))" under the underlined line. Additionally, you can go to the jsonl files generated by the data preparation script...

[BUG]: ColossalChat train sft is skipped with opt-1.3b model

## Data File Structure ## Content in Each Data File ## Data Preparation Script ## Tokenized Dataset File Structure Here, the arrow files are tokenized datafiles you need for the...