Support GPU accelerated inference for MacOS on apple silicon chips

[limingchina]

Is your feature request related to a problem? Please describe. It seems that currently only CPU is used for interference on Mac. I've tried my Mac M3 Pro.

Describe the solution you'd like It would be nice to support GPU accelerated inference for MacOS apple silicon chip.

Additional context Can you list the part where it needs to be modified to support that?

[aluminumbox]

There used to be PR using MPS, but it is slower than cpu on mac, maybe it is not mature yet

[lonelygo]

我fork 的版本提供了 macOS 下的支持，你可以试试看：https://github.com/lonelygo/CosyVoice

[jimersylee]

我fork 的版本提供了 macOS 下的支持，你可以试试看：https://github.com/lonelygo/CosyVoice

跑不起来 python webui.py

failed to import ttsfrd, use wetext instead Sliding Window Attention is enabled but not implemented for sdpa; unexpected results may be encountered. Failed to load model: problem in cosyvoice.flow.flow_matching - ImportError: cannot import name 'cached_download' from 'huggingface_hub' (/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/huggingface_hub/init.py) Traceback (most recent call last): File "/Users/jimmy/.local/share/uv/python/cpython-3.10.0-macos-aarch64-none/lib/python3.10/pydoc.py", line 435, in safeimport module = import(path) File "/Users/jimmy/projects/CosyVoice/cosyvoice/flow/flow_matching.py", line 17, in from matcha.models.components.flow_matching import BASECFM File "/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/matcha/models/components/flow_matching.py", line 6, in from matcha.models.components.decoder import Decoder File "/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/matcha/models/components/decoder.py", line 8, in from diffusers.models.activations import get_activation File "/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/diffusers/init.py", line 5, in from .utils import ( File "/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/diffusers/utils/init.py", line 36, in from .dynamic_modules_utils import get_class_from_dynamic_module File "/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/diffusers/utils/dynamic_modules_utils.py", line 28, in from huggingface_hub import HfFolder, cached_download, hf_hub_download, model_info ImportError: cannot import name 'cached_download' from 'huggingface_hub' (/Users/jimmy/projects/CosyVoice/.venv/lib/python3.10/site-packages/huggingface_hub/init.py)

[Stonedcs]

我fork 的版本提供了 macOS 下的支持，你可以试试看：https://github.com/lonelygo/CosyVoice

请问，rtf指标有多少提升？相比用CPU跑