paper-qa

paper-qa copied to clipboard

Have cheaper mode

6

Perhaps using `curie` or `sentence-transformer` for embeddings (relatively cheap anyway). Distillbert might be good too. https://huggingface.co/deepset/deberta-v3-large-squad2

whitead

whitead

https://github.com/jerryjliu/gpt_index/issues/440

whitead

These were missing. Should make them available again.

whitead

Any thoughts on OCR for older papers? (image-only)

7

EDIT: [a related OCR/NLP avenue](https://doi.org/10.48550/arXiv.2302.14045)

sgbaird

whitead

maybe_is_text() failed in Chinese or other language string

1

string.printable only check ASCII for other language, maybe decode with Unicode, but before that, should detect the encoding reference: https://chardet.readthedocs.io/en/latest/usage.html#basic-usage by the way, Is it necessary to check maybe_is_text()?

goldengrape

Nice project btw!

thiswillbeyourgithub

In case people want different caches or turn it off

whitead

What has to be the file format fed to docs.add

ManasiPat