binmakeswell comments

Results 359 comments of


                                            binmakeswell

[BUG]: stuck at initialization and no error message

> which example is related to this bug? gpt2 for the initial issue.

[FEATURE]: Expotential Moving Average (EMA)

Hi @taoisu , thanks for your idea. We have recorded your suggestion and will prioritize it later.

[FEATURE]: Can you push the Compared code in the readme?

Hi @wjizhong All example code can be found at https://github.com/hpcaitech/ColossalAI-Examples Other factors such as hardware architecture, network communication, and parameter settings all have a impact on performance, and even the...

[FEATURE]: require convenient hybrid_parallel example

ViT+CIFAR example will come soon.

[RFC] A tracer to monitor the memory usage during training

Work in progress

[DOC]: Need a simple Colab sample to show CUDA and Torch versions

Perhaps improvements are needed in the reqiuments and error message as well. Many features cannot be displayed in colab with one GPU, and ZeRO's compatibility with colab needs to be...

The performance of model parallelism (MP) is not good

Thanks for your report, detailed tests with stable code will come soon.

ZeRO dose not initialize weight correctly

Work in process

[BUG]: MoE example GroupNCCL cleanup error during training done and exiting CUDA

We have updated a lot. This issue was closed due to inactivity. Thanks.

Trying to adpat DLRM but encounter api incompatibility

We have updated a lot. This issue was closed due to inactivity. Thanks.