ServerlessLLM
Other hardware adaptation
If I wish to adapt sllm for the Ascend NPU, what modifications should I undertake?
Hi! Thanks for your interest in adapting SLLM for the Ascend NPU.
There’s already a version of SLLM that supports Ascend NPU available here: https://gitee.com/openeuler/ServerlessLLM . You can refer to that project for guidance and implementation details. If you encounter any issues or bugs, feel free to discuss them here — we’d be happy to help!
Hi! Thanks for your interest in adapting SLLM for the Ascend NPU.
There’s already a version of SLLM that supports Ascend NPU available here: https://gitee.com/openeuler/ServerlessLLM . You can refer to that project for guidance and implementation details. If you encounter any issues or bugs, feel free to discuss them here — we’d be happy to help!
Attempted to run the project on a CUDA-compatible metax GPU. The demo ran successfully, but when testing DeepSeekOCR, sllm-store segfaulted in model.cpp while copying the model to the GPU. Not sure if this is due to DeepSeekOCR not being supported yet.
