Langchain-Chatchat error while uploading pdf

[js333031]

RemoteProtocolError: API communication encountered an error: peer closed connection without sending complete message body

Full error logs are attached. Setup of the project is based on instructions provided at this repo. Instructions for entering issues there say to enter the issues here.

ipex-llm-langchain-chatchat.txt

[JinBridger]

Hi, @js333031,

We noticed that you put embedding model on xpu and llm on cpu. However, Langchain-chatchat is currently not supporting putting embedding model and llm on different devices. Could you please try to put embedding model and llm on the same device?

If you want to run langchain-chatchat on linux with Xeon CPU, please refer to this guide.

Please feel free to ask for any further problem : )

[js333031]

Thank you, that was partially the issue. With large file, another error happens. large-file-error.txt

Can you please tell me how to determine if AMX is being used on 4th Gen Xeon. That's what my system has and I would like to test with and without AMX using same precision in both cases. Can you please provide the steps?

[Oscilloscope98]

With large file, another error happens. large-file-error.txt

Hi @js333031,

To resolve this issue, you could try to decrease the "max length of each text" for text splitting: Or if there is a "Table of Contents" page in your PDF, you could also remove that page and have a try again as there could be some bug in text splitting for TOC with lots of ".........................".

Can you please tell me how to determine if AMX is being used on 4th Gen Xeon. That's what my system has and I would like to test with and without AMX using same precision in both cases. Can you please provide the steps?

AMX is automatically used for ipex-llm optimizations on 4th Gen Xeon.

Please let us know for any further questions :)