binmakeswell

icon indicating an email [email protected]

icon company @hpcaitech icon location Singapore icon location Stay Hungry, Stay Foolish.

Results 359 comments of binmakeswell

LAMB is not suited for tensor parallel

We have updated a lot. This issue was closed due to inactivity. Thanks.

[BUG]: stuck at initialization and no error message

We have updated a lot. This issue was closed due to inactivity. Thanks.

[FEATURE]: New way to tensotboard

We have updated a lot. This issue was closed due to inactivity. Thanks.

The performance of model parallelism (MP) is not good

We have updated a lot. This issue was closed due to inactivity. Thanks.

[BUG]: Different behaviors when setting param for colossalai.nn.Linear and torch.nn.Linear

We have updated a lot. This issue was closed due to inactivity. Thanks.

[BUG]: Zero returns fp16 tensors which causes RuntimeError

We have updated a lot. This issue was closed due to inactivity. Thanks.

[FEATURE]: require convenient hybrid_parallel example

We have updated a lot. This issue was closed due to inactivity. Thanks. https://github.com/hpcaitech/ColossalAI/tree/main/examples

[RFC] A tracer to monitor the memory usage during training

We have updated a lot. This issue was closed due to inactivity. Thanks.

[FEATURE]: Parallel layernorm optimization

We have updated a lot. This issue was closed due to inactivity. Thanks.

How PP and ZeRO stage 2+ work together?

We have updated a lot. This issue was closed due to inactivity. Thanks.