ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

torch.cuda.OutOfMemoryError: CUDA out of memory

Open cy565025164 opened this issue 1 year ago • 0 comments

🐛 Describe the bug

A10080G8卡的机器，batch_size=1，7B的llama-2模型，train_sft.py和train_reward_model.py都跑不起来

Environment

You are using a model of type mistral to instantiate a model of type llama. This is not supported for all configurations of models and can yield errors

cy565025164 avatar Nov 25 '23 13:11 cy565025164