[Question]: Deepdoc pdf parsing consumes too much memory

[DmSilkin]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (Language Policy).
• [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• [x] Please do not modify this template :) and fill in all the required fields.

Describe your problem

I've created a test pdf file with 5 MB size and 1600 pages inside. When I start to parse this file with deepdoc, task executor starts to consume a big amount of memory which eventually leads to oom.

Now I have almost 62gb ram and it is still not enough? Its just a simple 5mb file. How to fix this behaviour? How much memory does it need?

[dosubot[bot]]

Deepdoc's high memory usage comes from loading all PDF pages as images (at high resolution) and keeping them, plus all intermediate data (like OCR results and layout info), in memory for the entire parsing process. This means memory use scales with the number of pages and image resolution, not just the file size. For a 1600-page PDF, even 62GB RAM can be insufficient, especially with the default zoom factor and batch processing approach. There is no streaming or page-by-page memory release—everything is processed in large blocks and only released at the end, which can easily lead to out-of-memory errors for big documents (source).

To mitigate this, you can try: lowering the zoom/resolution parameter, limiting the number of pages parsed at once (if you can modify the code or split the document), or using the 'naive' or 'plain_text' parser for simple/text-based PDFs, which skips OCR and layout analysis and is much more memory-efficient (source). Reducing concurrency or batch size may also help. There are currently no built-in options for memory limiting or streaming in deepdoc, so resource management has to be handled externally (e.g., by splitting documents or tuning container limits) (source).

Some users report that memory is not fully released after parsing, so you may need to restart the server between large jobs (source). If you need to process very large PDFs regularly, consider breaking them into smaller chunks or requesting/implementing a streaming parsing feature.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[DmSilkin]

Can i run deepdoc on GPU? Will it be faster?

[dosubot[bot]]

Yes, you can run DeepDoc on GPU starting with RAGFlow v0.22.0. GPU acceleration is supported for OCR and layout analysis using ONNX Runtime with CUDAExecutionProvider, and DeepDoc will automatically detect and use available GPUs for these tasks if configured correctly. To enable this, set DEVICE=gpu in your Docker .env file before starting the service. You can also control GPU memory usage and parallelism with environment variables like OCR_GPU_MEM_LIMIT_MB and settings.PARALLEL_DEVICES for multi-GPU setups details code reference.

Running DeepDoc on GPU can make OCR and layout analysis significantly faster, especially for large or complex documents. However, overall speedup depends on your hardware and the specific bottlenecks in your workflow—other steps like image loading and text merging may still be CPU-bound deployment guide. There aren’t official benchmarks published, but user reports confirm noticeable speed improvements for the GPU-accelerated steps.

If you need even more performance, you can increase the OCR batch size (if your GPU has enough memory) and use multiple GPUs for parallel processing discussion. For distributed setups, RAGFlow can also offload parsing to remote GPU servers using MinerU.

Let me know if you want configuration examples or details on tuning for your hardware!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[KevinHuSh]

Can i run deepdoc on GPU? Will it be faster?

For open-source Deepdoc has nothing to do with GPU.