paper-qa icon indicating copy to clipboard operation
paper-qa copied to clipboard

maybe_is_text() failed in Chinese or other language string

Open goldengrape opened this issue 2 years ago • 1 comments

string.printable only check ASCII for other language, maybe decode with Unicode, but before that, should detect the encoding reference: https://chardet.readthedocs.io/en/latest/usage.html#basic-usage

by the way, Is it necessary to check maybe_is_text()?

goldengrape avatar Mar 05 '23 18:03 goldengrape

You can add disable_check=True to docs.add

whitead avatar Mar 05 '23 21:03 whitead