zhmzm issues

Repositories
Issues
Comments

Results 1 issues of


                                            zhmzm

DeepSpeed Zero-3 is not compatible with `low_cpu_mem_usage=True` or with passing a `device_map`

Hi, Thanks for sharing the code and models. I run the following command ``` master_port=18765 split=forget10 model=llama2-7b lr=2e-5 CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 --master_port=$master_port forget.py --config-name=forget.yaml split=${split} batch_size=4 gradient_accumulation_steps=4 model_family=${model} lr=${lr} ```...