text-generation-inference

text-generation-inference copied to clipboard

System Info

make install-server does not have Apple MacOS Metal Framework

• Please either remove from the readme info about brew/macOS altogether to not confuse users.
• OR add support for Apple MPS framework to the ./server folder so the make command can install on macOS.

Information

• [ ] Docker
• [X] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

#on macOS M-series: make install-server

Tries to install flash attention with NVIDIA CUDA

Expected behavior

make install-server should be optimized for Apple Metal Framework.