private-gpt icon indicating copy to clipboard operation
private-gpt copied to clipboard

fix: Support text files with other encodings than UTF-8

Open maozdemir opened this issue 1 year ago • 2 comments

Fixes #418 (#415). Fixes #453 As seen in langchain docs: https://github.com/hwchase17/langchain/blob/master/docs/use_cases/question_answering.md

maozdemir avatar May 23 '23 11:05 maozdemir

Maybe we should try this instead @maozdemir https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/file_directory.html?highlight=TextLoader#c-auto-detect-encodings

imartinez avatar May 23 '23 12:05 imartinez

Maybe we should try this instead @maozdemir https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/file_directory.html?highlight=TextLoader#c-auto-detect-encodings

Yes, my langchain was on 0.0.171 and apparently that feature was proposed in 0.0.174. Your suggestion works much better. @imartinez

maozdemir avatar May 23 '23 12:05 maozdemir