ColossalAI

ColossalAI copied to clipboard

[BUG]: loading OPT 66B model - CPU runs out of memory

2

### Is there an existing issue for this bug? - [X] I have searched the existing issues ### 🐛 Describe the bug I am trying to reproduce OPT-66B using 16xH100...

PurvangL

[FEATURE]: Support SP+PP in Llama etc.

1

### Describe the feature Currently most models like Llama does not support SP together with PP. Please add support for this.

GuangyaoZhang

[BUG]: Colossal AI failed to load ChatGLM2

2

### Is there an existing issue for this bug? - [X] I have searched the existing issues ### 🐛 Describe the bug I failed to run ChatGLM model with ColossalAI...

hiprince

### Describe the feature Please add Ulysses Sequence Parallelism support for Command-R, Qwen2 and ChatGLM

GuangyaoZhang

Use gemini plugin and LowLevelZero to run llama2_7b. In the pulgin in gemini, set the policy to static, shard_param_frac, offload_optim_frac, and offload_param_frac to 0.0, making gemini equal to zero2, and set stage to 2 in LowLevelZero. Using bf16 for training, and comparing the two plugins, we found that the GPU memory usage of gemini is higher than that of LowLevelZero. Why is this? In principle, gemini should save more GPU memory

2

JJGSBGQ

[FEATURE]: LoRA with sharded model

1

### Describe the feature Hi, when training big model like llama2-70b with lora, it will run into oom due to the unsharded model. It could help a lot if lora...

KaiLv69

Gradients are None after booster.backward

10

After calling booster.backward(loss=loss, optimizer=optimizer), all gradients of model.module are None. Is there a way to access the gradients?

ArnaudFickinger

[gemini] support lora

1

Fridge003

## 📝 What does this PR do? - Pytorch 2.3.0 added a `group` argument in `_object_to_tensor` function. Updated related use cases in pipeline communication.

kurisusnowdeng

1. Optimize the data path: from `List->CPU Tensor->List->rpc_param->GPU Tensor` to `List->rpc_param->GPU Tensor` 2. Wrap the async forward only once 3. Only rank0 Worker runs the sampler and returns the return...

LRY89757