Implements PDF.js as a primary PDF extraction library

[orfin]

This PR makes PDF.js to be used as a default PDF converting library. That solves #138 and allows to run the library e.g. on AWS Lambda or environments that we don't want to pollute with external binaries.

It was written as a drop-in replacement for pdftotext, so all previous unit tests are passing -- however that means inside PDF.js we have funky "pdftotextOptions" assignments.

You are still able to use pdftotext when providing application/pdf-pdftotext as a typeOverride parameter - @dbashford we were thinking about the best way to have possibility to keep both libraries and chose that way. Maybe you see another way to do it more properly? :-)

Performance comparison

@dbashford you were concerned about performance of PDF.js as it's purely javascript based. We made comparison between pdf.js and pdftotext and it seems that PDF.js may be slightly slower, however the difference is not huge.

Small PDF

$ time ./bin/textract test/files/testpdf-multiline.pdf
This is a test,
A multi-line test,
Lets hope it works

real	0m1.093s
user	0m0.965s
sys	0m0.122s

$ time ./bin/textract test/files/testpdf-multiline.pdf --typeOverride=application/pdf-pdftotext
This is a test,
A multi-line test,
Lets hope it works

real	0m0.891s
user	0m0.766s
sys	0m0.113s

Bigger PDF

$ time ./bin/textract test/files/two_columns.pdf

real	0m1.556s
user	0m1.673s
sys	0m0.145s

$ time ./bin/textract test/files/two_columns.pdf --typeOverride=application/pdf-pdftotext
real	0m0.988s
user	0m0.854s
sys	0m0.121s

[dbashford]

This is awesome. Literally 10 minutes before I saw the email for this I was thinking "I really need to spend some time looking into pdf.js".

I'll take a hard look at this, get this merged, touch up a few things and hopefully get it out with some spare time I have later today.

Thanks!!

[orfin]

Great to hear that and looking forward to your comments :-)

BTW. I'll be also working on implementing js-native extraction for DOC and PPT documents, potentially using following libraries:

• doc - https://github.com/morungos/node-word-extractor
• ppt - https://github.com/SheetJS/js-ppt

However, I saw that ppt code is already in textract, but it's commented: https://github.com/dbashford/textract/blob/master/lib/extractors/ppt.js what's the reason? ppt lib didn't work well?

[dbashford]

Yeah, the ppt library didn't work out from the beginning. 4 years ago I gave it a shot and that library just hasn't been updated. I kept the code in there to remind myself I gave it a shot.

[dbashford]

Heads up, trying to get a smaller bug fix release out before I include this as I suspect including this will require a major bump in version. If it works, not sure I really care to continue to support pdftotext, can push to 3.0 and leave it behind. Before something that drastic, though, want to get a few tiny things done and get a point release out.

(Hence the merge issues, apologies!)

[ghost]

Is there an update on this? Is there a fork that uses pdf.js ?

[Shing-Ho]

https://github.com/Shing-Ho/textract use this one. This doesn't require pdftotext and antiword