binmakeswell comments

Results 359 comments of


                                            binmakeswell

[BUG]: gpt example auto_parallel requirements.txt bug

Hi @l241025097 You are right! Could you please submit a PR to fix it? Thank you very much for your contribution!

I want to implement stage1 in chatGPT. What should I do？

This issue was closed due to inactivity. Thanks.

[BUG]: errors of combine shard_init and tensor parallel in rlhf examples

Hi @ziyuwan @taishiciR As mentioned in [Chat example](https://github.com/hpcaitech/ColossalAI/tree/main/applications/Chat/examples), --strategy 'colossalai_gemini' or 'colossalai_zero2' is enough for most cases. TP is not supported for Chat currently. It is relatively low on our...

[BUG]: Pipeline Parallel in ChatGPT examples

Hi @evi-Genius @taishiciR As mentioned in [Chat example](https://github.com/hpcaitech/ColossalAI/tree/main/applications/Chat/examples), –strategy ‘colossalai_gemini’ or ‘colossalai_zero2’ is enough for most cases. PP is not supported for Chat currently. It is relatively low on our...

prevent op_builder being installed in site-pkgs

Hi @jeffra Thank you so much for your contribution!

[BUG]: 单机单卡和单机八卡训练时间一样

This issue was closed due to inactivity. If you have further questions, please [open another new issue](https://github.com/hpcaitech/ColossalAI/issues/new/choose) and provide details. Thanks.

Small memory saving on NVME[BUG]:

> @ver217 I trained opt 66b on the single A100. Hi @MikeChenfu 66b is much larger than the capacity of one A100 80GB memory. This issue was closed due to...

AttributeError: _old_init is not found in the LitEma, please make sure that you have imported LitEma before entering the context.

Hi @guohe369 We have updated a lot. Please check the latest code. This issue was closed due to inactivity. Thanks.

In this code line - trainer.fit(dataset, ...), What is the dataset need to be considered.

Hi @Pradeep-Vanapalli We have updated a lot. Please check the latest code. This issue was closed due to inactivity. Thanks.

The divisible factor of local world size in sequence parallelism.

Hi @GeneZC Welcome to share your findings with new issues or discussion. This issue was closed due to inactivity. Thanks.