pdfminer icon indicating copy to clipboard operation
pdfminer copied to clipboard

PDFDocument is slow, any way to speed it up?

Open typhoon71 opened this issue 5 years ago • 4 comments

I'm currently doing this:

fp = open(pdf_fpath, 'rb')
parser = PDFParser(fp)
doc = PDFDocument(parser)

With the pdfs I'm working with, PDFDocument is really too slow. Is there any way to speed it up?

I see that there's often the suggestion to "giving -n option which turns off automatic layout analysis.", but I have no idea how to do that from python.

I "only" need to read xrefs and objects/streams, I don't care how the pdf would be rendered/pages. Can anyone help please? Thanks.

typhoon71 avatar Jan 06 '20 22:01 typhoon71

From that example:

parser = PDFParser(in_file)
doc = PDFDocument(parser)

was exactly where I timed the slowdown.

 doc = PDFDocument(parser)

is time consumnig, like 99%.

If I understand your suggestion, I should make my own version of PDFDocument? One that does not analyze the layout?

No time for it right now, but maybe I could in the future.

Actually I hoped in some kind of switch for PDFDocument... but I found nothing.

typhoon71 avatar Jan 07 '20 18:01 typhoon71

I'm sorry. My comment was short, not clear and partially wrong. I've deleted it.

Layout analysis is not used during the parsing of the PDF. It is only used when you interpret the document, with e.g. TextConverter of HTMLConverter.

Could you share an example PDF that PDFDocument is really slow on?

pietermarsman avatar Jan 07 '20 20:01 pietermarsman

Sadly I can't share those PFDs. The thing is that once I got PDFDocument initialized, everything else is really fast. Maybe it's decompressing the PDF, that could be slowing it.

typhoon71 avatar Jan 09 '20 20:01 typhoon71

import fitz

filepath = "C:\\user\\docs\\aPDFfile.pdf"

text = ''
with fitz.open(filepath ) as doc:
    for page in doc:
        text+= page.get_text()
print(text)

hiDaDeng avatar Dec 11 '21 03:12 hiDaDeng