[Question]: Parsing file is too slow.

[ctjian]

Describe your problem

如图，一直在解析中

[KevinHuSh]

Is it on the demo or locally deployed? Cancel it and redo.

[ctjian]

Is it on the demo or locally deployed? Cancel it and redo.

yes, it is locally deployed. Redo still doesn't work.

[ShaswatPanda]

@ctjian deploy with GPU; parsing is exponentially faster. Change your docker-compose to this:

include:
  - path: ./docker-compose-base.yml
    env_file: ./.env

services:
  ragflow:
    depends_on:
      mysql:
        condition: service_healthy
      es01:
        condition: service_healthy
    image: infiniflow/ragflow:${RAGFLOW_VERSION}
    container_name: ragflow-server
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            device_ids: ['0']
            capabilities: [gpu]
    ports:
      - ${SVR_HTTP_PORT}:9380
      - 80:80
      - 443:443
    volumes:
      - ./service_conf.yaml:/ragflow/conf/service_conf.yaml
      - ./ragflow-logs:/ragflow/logs
      - ./nginx/ragflow.conf:/etc/nginx/conf.d/ragflow.conf
      - ./nginx/proxy.conf:/etc/nginx/proxy.conf
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
    environment:
      - TZ=${TIMEZONE}
      - HF_ENDPOINT=https://huggingface.co
    networks:
      - ragflow
    restart: always

[ctjian]

@ctjian deploy with GPU; parsing is exponentially faster. Change your docker-compose to this:

include:
  - path: ./docker-compose-base.yml
    env_file: ./.env

services:
  ragflow:
    depends_on:
      mysql:
        condition: service_healthy
      es01:
        condition: service_healthy
    image: infiniflow/ragflow:${RAGFLOW_VERSION}
    container_name: ragflow-server
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            device_ids: ['0']
            capabilities: [gpu]
    ports:
      - ${SVR_HTTP_PORT}:9380
      - 80:80
      - 443:443
    volumes:
      - ./service_conf.yaml:/ragflow/conf/service_conf.yaml
      - ./ragflow-logs:/ragflow/logs
      - ./nginx/ragflow.conf:/etc/nginx/conf.d/ragflow.conf
      - ./nginx/proxy.conf:/etc/nginx/proxy.conf
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
    environment:
      - TZ=${TIMEZONE}
      - HF_ENDPOINT=https://huggingface.co
    networks:
      - ragflow
    restart: always

Thank you. Problem solved.

[Mr-Moonsilver]

🚀 Enable GPU Acceleration in RAGFlow (Linux Mint 21 / Ubuntu 22.04)

Tested May 2025 on Linux Mint 21 (Jammy base) + RTX 3090 + NVIDIA driver 560 + CUDA 12.6
Based on GitHub issue #1019.

---

0. Prerequisites

Item	Version tested	Watch out-for
NVIDIA driver	560.35.03	nvidia-smi must work on the host before Docker.
CUDA runtime	12.6 (comes with the driver)	CUDA ≤12.1? use the matching torch wheel (see step 5).
Docker	24 +	sudo docker run hello-world should succeed.
Docker Compose	v2 plug-in	Mint ships the plug-in; no docker-compose binary needed.

---

1. Install NVIDIA Container Toolkit on Mint 21

Mint identifies as linuxmint21, and NVIDIA’s apt repos are… messy.
Solution: point apt to the Ubuntu 18.04 path (works for 22.04/Mint 21) and use its key.

sudo mkdir -p /usr/share/keyrings
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/amd64/ /" \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit

Watch out-fors

• 404/Release errors → you copied the wrong path (stick to ubuntu18.04/amd64).
• If apt install complains about unsigned keys, redo the curl | gpg --dearmor step.

Enable it for Docker:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Quick sanity check:

docker run --rm --gpus all nvidia/cuda:12.0.1-base-ubuntu22.04 nvidia-smi

2. Patch kernel sysctl (Valkey + ES want these)

echo 'vm.overcommit_memory = 1' | sudo tee -a /etc/sysctl.conf
echo 'vm.max_map_count    = 262144' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

3. Clone RAGFlow & switch to the docker folder

git clone https://github.com/infiniflow/ragflow.git
cd ragflow/docker

Make a logs dir so they persist:

mkdir ragflow-logs

4. Create a CUDA-enabled server image

Why?

infiniflow/ragflow:* images are CPU-only (no torch).
We build a derivative that injects CUDA torch 2.7 + cu126.

4.1 Dockerfile.gpu

# Dockerfile.gpu
ARG RAGFLOW_IMAGE
FROM ${RAGFLOW_IMAGE}
Bootstrap pip into the venv used by ragflow-server
RUN python3 -m ensurepip --upgrade 

&& python3 -m pip install --upgrade pip setuptools wheel
Install CUDA-enabled PyTorch (match wheel to your CUDA runtime)
RUN python3 -m pip install --no-cache-dir 

torch==2.7.0+cu126 torchvision==0.22.0+cu126 torchaudio==2.7.0+cu126 

--extra-index-url https://download.pytorch.org/whl/cu126

4.2 docker-compose-gpu.yml

include:
  - ./docker-compose-base.yml
services:
ragflow:
depends_on:
mysql:
condition: service_healthy
build:
context: .
dockerfile: Dockerfile.gpu
args:
RAGFLOW_IMAGE: ${RAGFLOW_IMAGE}   # e.g. v0.18.0-slim in your .env
image: ragflow-server-gpu:local
container_name: ragflow-server
ports:
- "${SVR_HTTP_PORT}:9380"
- "80:80"
- "443:443"
volumes:
- ./ragflow-logs:/ragflow/logs
- ./nginx/ragflow.conf:/etc/nginx/conf.d/ragflow.conf
- ./nginx/proxy.conf:/etc/nginx/proxy.conf
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
env_file: .env
environment:
- TZ=${TIMEZONE}
- HF_ENDPOINT=${HF_ENDPOINT}
- MACOS=${MACOS}
networks:
- ragflow
restart: on-failure
extra_hosts:
- "host.docker.internal:host-gateway"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]

5. Build & run the stack

docker compose -f docker-compose-gpu.yml down            # stop any old containers
docker compose -f docker-compose-gpu.yml build --no-cache --pull
docker compose -f docker-compose-gpu.yml up -d

Build-time gotchas

Error	Fix
No module named pip	we added python3 -m ensurepip --upgrade
torch whl not found	pick the wheel that matches your host CUDA (cu116, cu118, cu121, cu126…)

6. Confirm torch + CUDA inside the container

docker exec -it ragflow-server python3 - 

Expected:

CUDA available = True
GPU : NVIDIA GeForce RTX 3090

Allocate a tensor to see VRAM bump:

docker exec ragflow-server python3 -c "import torch, time; x=torch.randn(4096,4096,device='cuda'); print('done'); time.sleep(10)"
# then in another shell:
watch -n1 nvidia-smi

7. Wire up RAGFlow parsing / embeddings on GPU

UI → Knowledge Base → Parser mode → choose DeepDoc (GPU).

Embedding model: leave at default or point to a local Ollama/vLLM endpoint.

Upload a PDF and watch GPU util spike (watch -n1 nvidia-smi).

No more can’t import package ‘torch’ logs.

9. Troubleshooting checklist

Symptom	Fix
ModuleNotFoundError: torch inside container	Rebuild image; ensure python3 -m pip install … not pip
nvidia-smi inside container shows ‘No devices’	nvidia-ctk runtime configure --runtime=docker and restart Docker
GPU idle but CPU pegged	confirm you selected DeepDoc (GPU) or embeddings are on GPU
Build hangs pulling torch wheels	use a mirror or pre-download wheels into build context