[Question]: How do I build an image that runs on GPU?

[Ykubernetes]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (Language Policy).
• [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• [x] Please do not modify this template :) and fill in all the required fields.

Describe your problem

To avoid downloading dependencies and rebuilding an image suitable for GPU operation every time a container is started How do I build an image that runs on GPU? What dependencies are needed? What files need to be modified? What are the build documentation?

[dosubot[bot]]

To build a RAGFlow Docker image that runs on GPU, you need to ensure both your host and your image are set up for GPU support:

Host prerequisites:

• Install the NVIDIA driver and NVIDIA Container Toolkit on your host.
• Test your setup with:

  docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
  

  If this works, your Docker is GPU-ready [source].

Image and configuration:

• Use or build the official RAGFlow Docker image (from v0.22.0, only the slim edition is released, ~2GB, x86 only) [source].
• For GPU support in DeepDoc tasks, set DEVICE=gpu in your .env file before starting the container [source].
• Start with:

  docker compose -f docker-compose.yml up -d
  

• If you want to use Docker Compose's explicit GPU support, use or adapt docker-compose-gpu.yml (see the deploy.resources.reservations.devices section with driver: nvidia, count: all, capabilities: [gpu]) [source].

Dependencies:

• Make sure your image includes GPU-enabled libraries: onnxruntime-gpu, fastembed-gpu, and PyTorch with CUDA (not the CPU-only version). If you use MinerU, install it without the CPU-only PyTorch wheel index URL [source].
• Check pyproject.toml for these dependencies if you are customizing the build [source].

Files to modify:

• Dockerfile: Ensure it installs the GPU-enabled versions of dependencies.
• .env: Set DEVICE=gpu and, if using MinerU, USE_MINERU=true and relevant MinerU variables.
• docker-compose-gpu.yml or docker-compose.yml: Add the GPU device reservation section if not present.

Scope of GPU acceleration:

• RAGFlow itself mostly uses CPU for document parsing; GPU is mainly used for embedding/LLM inference. For full GPU acceleration, deploy an embedding inference service (like Ollama, Xinference, or vLLM) on GPU and point RAGFlow to it [source].

Docs and references:

• Official Docker Compose GPU support
• RAGFlow Docker build docs

Let me know if you need a sample Dockerfile snippet or more details for a specific use case.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[KevinHuSh]

The GPU is not necessary and has little thing to do with accelerating RAGFlow.