infiniflow
[Bug]: deepdoc use GPU still appears this error
Is there an existing issue for the same bug?
- [x] I have checked the existing issues.
RAGFlow workspace code commit ID
29fb6ca11b318bd437b82299a800aca91f679efa
RAGFlow image version
v0.16.0
Other environment information
- GPU: A800 80G
- CUDA Version: 12.4
- OS: Ubuntu 20.04.1
- Docker Version: 24.0.7
Actual behavior
knowledge >> pdf file >> recognizer error
Expected behavior
Traceback (most recent call last): File "/ragflow/rag/svr/task_executor.py", line 626, in handle_task do_handle_task(task) File "/ragflow/rag/svr/task_executor.py", line 559, in do_handle_task chunks = build_chunks(task, progress_callback) File "/ragflow/rag/svr/task_executor.py", line 218, in build_chunks cks = chunker.chunk(task["name"], binary=binary, from_page=task["from_page"], File "/ragflow/rag/app/manual.py", line 190, in chunk sections, tbls = pdf_parser(filename if not binary else binary, File "/ragflow/rag/app/manual.py", line 54, in call self._layouts_rec(zoomin) File "/ragflow/deepdoc/parser/pdf_parser.py", line 327, in _layouts_rec self.boxes, self.page_layout = self.layouter( File "/ragflow/deepdoc/vision/layout_recognizer.py", line 70, in call layouts = super().call(image_list, thr, batch_size) File "/ragflow/deepdoc/vision/recognizer.py", line 487, in call bb = self.postprocess(self.ort_sess.run(None, {k:v for k,v in ins.items() if k in self.input_names}, self.run_options)[0], ins, thr) File "/ragflow/.venv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run return self._sess.run(output_names, input_feed, run_options) onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Softmax node. Name:'/model.23/dfl/Softmax' Status Message: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:376 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*, bool, onnxruntime::WaitNotificationFn) Available memory of 0 is smaller than requested bytes of 5505024
Steps to reproduce
knowledge >> pdf file >> recognizer error
Additional information
No response
It seems short of memory. OOM.
- docker-compose-gpu.yml Specify id:3 gpu
- when upload .pdf to knowledge and start embedding, I see that the CPU still has enough memory.
- I don't know why this step why use show much gpu memory? File "/ragflow/deepdoc/vision/recognizer.py", line 487, in call bb = self.postprocess(self.ort_sess.run(None, {k:v for k,v in ins.items() if k in self.input_names}, self.run_options)[0], ins, thr)
As I know, onnxruntime did not support GPU well
now, Is it recommended to use CPU mode?
I also encountered this problem
rag flow version
v0.16.0-24
error message
onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*, bool, onnxruntime::WaitNotificationFn) Available memory of 0 is smaller than requested bytes of 5505024
environment
NVIDIA A100 80GB
I also encountered this problem
rag flow version
v0.16.0-24
error message
onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*, bool, onnxruntime::WaitNotificationFn) Available memory of 0 is smaller than requested bytes of 5505024
environment
NVIDIA A100 80GB
I can successfuly use cpu to extract pdf or other types of file!
I also encountered this problem. Uploading a PDF to start embedding. I observe that one GPU is being used. However, in “docker-compose-gpu.yml“, the count is set to all.
