大佬 7b bloom 大约需要多大的内存在非lora 的情况下，单机两张24 3090可以吗？

[liuliuNLP]

[xianghuisun]

应该是不够的。建议您可以尝试cpu offload模式

[liuliuNLP]

{ "train_batch_size": "auto", "optimizer": { "type": "Adam", "params": { "lr": "auto", "betas": [ 0.9, 0.999 ], "eps": "auto", "weight_decay": "auto" } }, "overwrite": true, "steps_per_print": 5, "fp16": { "enabled": true, "min_loss_scale": 1, "opt_level": "O2" }, "zero_optimization": { "stage": 2, "offload_optimizer": { "device": "cpu", "pin_memory": true }, "allgather_partitions": true, "allgather_bucket_size": 5e8, "overlap_comm": true, "reduce_scatter": true, "reduce_bucket_size": 5e8, "contiguous_gradients": true }, "scheduler": { "type": "WarmupLR", "params": { "warmup_min_lr": "auto", "warmup_max_lr": "auto", "warmup_num_steps": "auto" } } } 您好，这个是我加的cpu offload模式，但是要求我内存必须256G，这个是不是要扩充

[answerMA]

哈喽兄弟， 你这个配置文件能在3090上跑起来吗

[answerMA]

{ "train_batch_size": "auto", "optimizer": { "type": "Adam", "params": { "lr": "auto", "betas": [ 0.9, 0.999 ], "eps": "auto", "weight_decay": "auto" } }, "overwrite": true, "steps_per_print": 5, "fp16": { "enabled": true, "min_loss_scale": 1, "opt_level": "O2" }, "zero_optimization": { "stage": 2, "offload_optimizer": { "device": "cpu", "pin_memory": true }, "allgather_partitions": true, "allgather_bucket_size": 5e8, "overlap_comm": true, "reduce_scatter": true, "reduce_bucket_size": 5e8, "contiguous_gradients": true }, "scheduler": { "type": "WarmupLR", "params": { "warmup_min_lr": "auto", "warmup_max_lr": "auto", "warmup_num_steps": "auto" } } } 您好，这个是我加的cpu offload模式，但是要求我内存必须256G，这个是不是要扩充

哈喽兄弟， 你这个配置文件能在3090上跑起来吗