RuntimeError: CUDA error: invalid device ordinal

Open TranscenderNing opened this issue 2 years ago • 1 comments

在运行器件报错：Traceback (most recent call last): File "/home/ldn/ChatGLM130Dir/GLM-130B/generate.py", line 218, in args = initialize(extra_args_provider=add_generation_specific_args) File "/home/ldn/ChatGLM130Dir/GLM-130B/initialize.py", line 51, in initialize args = get_args(args_list) File "/home/ldn/anaconda3/envs/glm1301/lib/python3.9/site-packages/SwissArmyTransformer/arguments.py", line 385, in get_args initialize_distributed(args) File "/home/ldn/anaconda3/envs/glm1301/lib/python3.9/site-packages/SwissArmyTransformer/arguments.py", line 414, in initialize_distributed torch.cuda.set_device(args.device) File "/home/ldn/anaconda3/envs/glm1301/lib/python3.9/site-packages/torch/cuda/init.py", line 314, in set_device torch._C._cuda_setDevice(device) RuntimeError: CUDA error: invalid device ordinal CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Jun 14 '23 13:06 TranscenderNing

same

Jun 15 '23 16:06 alexhmyang