[BUG/Help] M1上无法运行，报错“AssertionError: Torch not compiled with CUDA enabled”

[liudaolunboluo]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

本地下载完成模型，修改完代码，运行python cli_demo.py，报错AssertionError: Torch not compiled with CUDA enabled，似乎是cuda不支持arm架构，本地启了一个conda装了pytorch，但是不能装cuda

Expected Behavior

No response

Steps To Reproduce

1、python cli_demo.py

Environment

- OS: mac 13.0
- Python: 3.9.12
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : false

Anything else?

No response

[liudaolunboluo]

把代码里的cuda()去掉，报错：

RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

[LixinLu42]

see the CPU部署 of readme

[liudaolunboluo]

see the CPU部署 of readme

.cuda()更换为了.to('mps')，并且按照apple官网上的文档用pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu安装了PyTorch-Nightly，能给进入到用户输入环节，但是等待响应的时候报错：

loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/97f6331a-ba75-11ed-a4bc-863efbbaf80d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x4x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).

[DingBool]

see the CPU部署 of readme

.cuda()更换为了.to('mps')，并且按照apple官网上的文档用pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu安装了PyTorch-Nightly，能给进入到用户输入环节，但是等待响应的时候报错：

loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/97f6331a-ba75-11ed-a4bc-863efbbaf80d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x4x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).

我是根据 https://pytorch.org/ 官网去安装 并且下载了 Hugging Face .float().to('mps') 如果运行不起来的话就得改成 .float()

[liudaolunboluo]

see the CPU部署 of readme

.cuda()更换为了.to('mps')，并且按照apple官网上的文档用pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu安装了PyTorch-Nightly，能给进入到用户输入环节，但是等待响应的时候报错：

loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/97f6331a-ba75-11ed-a4bc-863efbbaf80d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x4x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).

我是根据 https://pytorch.org/ 官网去安装 并且下载了 Hugging Face .float().to('mps') 如果运行不起来的话就得改成 .float()

你是M1吗？好像M1不能half，但是如果用float的话就OOM内存不足了😭

[DingBool]

see the CPU部署 of readme

.cuda()更换为了.to('mps')，并且按照apple官网上的文档用pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu安装了PyTorch-Nightly，能给进入到用户输入环节，但是等待响应的时候报错：

loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/97f6331a-ba75-11ed-a4bc-863efbbaf80d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x4x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).

我是根据 https://pytorch.org/ 官网去安装 并且下载了 Hugging Face .float().to('mps') 如果运行不起来的话就得改成 .float()

你是M1吗？好像M1不能half，但是如果用float的话就OOM内存不足了😭

m1 16G 只能 .float() 跑 但是问答需要很久 我现在已经放弃了 等更低的模型在来尝试