datalab-to
Performance issues despite using GPU/Text recognition chunk inconsistency
having performance issues using marker-server on windows 11, with Nvidia GPU, (primarily with obsidian via plugin "OCR AI" but even using marker_single from cli results in about the exact same performance.)
for very small documents it performs decently enough, but i am trying to convert a 500 page document to markdown, it is 2% done after almost 2 hours (ranging 1.6 to 30s/it) , i have many documents to convert and i am wondering if there is a way to make it perform faster or if there are any improvements that could be made in future version to help older / lower end hardware have a chance. possibly the addition of a slightly lower accuracy mode that minimizes resource load or lighter models or something? perhaps a mode to just fast run the OCR portion with a very small model, then scan for errors?
hardware specs: CPU: i7-7820x 8c/16t RAM: 96GB GPU1: intel arc A310 4GB GPU2: Nvidia Tesla P4 8GB (comparable to a GTX 1080, cuda toolkit 12.6, driver version: 573.48, compute compatibility 6.1, completely dedicated to the AI task and running nearly up to date everything.)
packages: torch 2.9.0+cu126, marker-pdf 0.2.17, surya-ocr 0.17.0
after playing with this some more, with different documents, it seems to really slow down in the text recognition phase and often it seems it is because it hangs on different chunks of the text recognition.
i have given it 300 mostly single page documents to process, mostly only a paragraph or two of text per document, leaving it for last night and all of today, it has finished 13 documents. some finished in 20-30 seconds others took 1-3 hours or longer, one took 9.5 hours. watching it here and there i notice it hangs on certain chunks, the 9.5 hour one for example, it processed most the text in about 10 seconds then proceeded to hang on chunk 12/13 for almost 10 hours.
a lot of the performance issues seem to be the fact that chunk processing is very inconsistent and there seems to be a need for optimization within start up time and consistency.
(although text recognition is still a bit slow on older hardware taking roughly 10 seconds up to a minute or more per page when it doesn't hang at all. but that is at least half way bearable compared to the hanging for random unknowns periods that can be anywhere from a few minutes to hours.)
So far my solution to this has been to switch to Mineru which performs the text recognition phase 500x-65,000x faster (not joking, this tool can take 500s or more per iteration and mineru hits up to 130it/s, so it does in 1/130th of a second what takes this one 5-10 minutes or more, not sure which is better exactly, i would like to use this one, but i cant wait up to hours on a single document.)