Exited with a "input types are not broadcast compatible" on Mac M1Max

[edwios]

Describe the bug

Using the default settings, text-generation-webui exited with an LLVM error when Generate button was clicked.

using option --cpu avoided this but speed was very slow at 1.15 tokens/s.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

On a M1 Mac, run: python server.py

Screenshot

No response

Logs

$ python server.py 
Loading pygmalion-6b...
Loading checkpoint shards: 100%|██████████████████| 2/2 [00:02<00:00,  1.31s/it]
Loaded the model in 4.36 seconds.
/Users/soro/miniforge3/envs/pytorch-m1/lib/python3.9/site-packages/gradio/deprecation.py:40: UserWarning: The 'type' parameter has been deprecated. Use the Number component instead.
  warnings.warn(value)
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
/Users/soro/miniforge3/envs/pytorch-m1/lib/python3.9/site-packages/transformers/generation/utils.py:686: UserWarning: MPS: no support for int64 repeats mask, casting it to int32 (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/mps/operations/Repeat.mm:236.)
  input_ids = input_ids.repeat_interleave(expand_size, dim=0)
loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/9e200cfa-7d96-11ed-886f-a23c4f261b56/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x22x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).
Abort trap: 6
/Users/soro/miniforge3/envs/pytorch-m1/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

System Info

Apple MacBook Pro M1 Max, 64GB RAM, MacOS 13.2.1

Python 3.9.13
accelerate                    0.17.1
torch                         2.0.0.dev20230301
transformers                  4.28.0.dev0