ragflow [Question]: onnxruntime::WaitNotificationFn) Available memory of 0 is smaller than requested bytes of 175657216

Self Checks

[x] I have searched for existing issues search for existing issues, including closed ones.
[x] I confirm that I am using English to submit this report (Language Policy).
[x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
[x] Please do not modify this template :) and fill in all the required fields.

Describe your problem

解析PDF文件以及其他文件报错过程如下：

开始于: Tue, 13 May 2025 14:05:53 GMT 持续时间: 86.75 s 进度: 14:05:53 Task has been received. 14:05:56 Page(1~13): Start to parse. 14:05:56 Page(1~13): Visual model detected. Attempting to enhance figure extraction... 14:05:56 Page(1~13): OCR started 14:06:24 Page(1~13): [ERROR]Internal server error while chunking: Exceptions from Trio nursery (2 sub-exceptions) 14:06:24 [ERROR][Exception]: Exceptions from Trio nursery (2 sub-exceptions) -- [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Conv node. Name:'p2o.Conv.28' Status Message: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:376 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*, bool, onnxruntime::WaitNotificationFn) Available memory of 0 is smaller than requested bytes of 187085056 14:05:53 Task has been received. 14:06:52 Page(13~21): Start to parse. 14:06:52 Page(13~21): Visual model detected. Attempting to enhance figure extraction... 14:06:52 Page(13~21): OCR started 14:07:15 Page(13~21): [ERROR]Internal server error while chunking: Exceptions from Trio nursery (2 sub-exceptions) 14:07:16 [ERROR][Exception]: Exceptions from Trio nursery (2 sub-exceptions) -- [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Conv node. Name:'p2o.Conv.28' Status Message: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:376 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*, bool, onnxruntime::WaitNotificationFn) Available memory of 0 is smaller than requested bytes of 175657216

May 13 '25 08:05 kou1584423647

To address the ONNXRuntime memory allocation error you're encountering during PDF parsing in RAGFlow, you can try the following solutions:

Limit GPU Memory Usage: If you're using a GPU, configure ONNX Runtime to limit GPU memory usage. You can do this by setting specific options, such as:

cuda_provider_options = {
    "device_id": device_id,
    "gpu_mem_limit": 512 * 1024 * 1024,  # Adjust the limit as needed
    "arena_extend_strategy": "kNextPowerOfTwo"
}

Deploy on CPU: It is recommended to run the RAGFlow server on a CPU instead of a GPU. GPU acceleration is not necessary for the RAGFlow server itself and should be reserved for the embedding inference server to improve chunking speed. This approach avoids CUDA memory allocation issues when running the Conv node on a GPU.
Upgrade ONNXRuntime Packages: If you encounter the ONNXRuntime GPU memory allocator error during PDF parsing, consider upgrading both onnxruntime and onnxruntime-gpu packages from version 1.17.1 to version 1.18.0. This upgrade should help resolve the error and allow PDF parsing to work correctly ^[1]^[2]^[3]^[4].