[Question]: Deployment of RAGFlow GPU Version on Hygon Computing Card Cluster

[luboyan6]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (Language Policy).
• [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• [x] Please do not modify this template :) and fill in all the required fields.

Describe your problem

Does anyone have experience deploying the GPU version of RAGFlow using Hygon computing cards? I'd like to ask for some advice.

[dosubot[bot]]

There’s currently no official or community-documented support for deploying the GPU version of RAGFlow on Hygon computing cards. All GPU deployment instructions and Docker images are designed for NVIDIA GPUs with CUDA, and there’s no mention of Hygon or alternative GPU hardware in the docs, code, or recent updates (source).

If your Hygon cards support CUDA and PyTorch, you could try building custom Docker images with the necessary GPU libraries (onnxruntime-gpu, fastembed-gpu, PyTorch with CUDA) and set DEVICE=gpu in your .env file. However, compatibility is not guaranteed, and you may encounter issues similar to those reported with newer or non-standard GPU hardware (source).

If you do manage to get it working, sharing your experience would help others in the community. For now, keep an eye out for future updates or community contributions that might add support for alternative GPU hardware.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Magicbook1108]

I’m afraid I’m unable to help with this issue, as it’s beyond my current ability. However, I’m required to respond to every question within 24 hours, which is why you’re seeing this message. If someone later provides useful information, this message will be removed.

[KevinHuSh]

GPU has nothing to do with RAGFlow. You could deploy embedding inference service on GPU which accelerates indexing and searching procedure.

[luboyan6]

GPU has nothing to do with RAGFlow. You could deploy embedding inference service on GPU which accelerates indexing and searching procedure.GPU 和 RAGFlow 没关系。你可以在 GPU 上部署嵌入推理服务，加快索引和搜索过程。

@KevinHuSh I’ve just started working on this project—could you please provide a more detailed plan? Thank you very much!