HKUDS
When pushing many files, the ui hangs
The ui is freezing and I can see that the backend is still processing the files:
When I reboot the backend, the same status can be found while this time nothing is happening on the backend.
Mayebe the file upload status must be recovered after reboot, we must fix the freezing. And also the import is taking way much more time than before, We need to add some debug information to see where there is a problem. I suspect that the LLM is failing to generate correct output which results in hanging nearely forever.
@danielaskdd @ArnoChenFx , can you please take a look at that?
What happen if you trigger SCAN from WebUI after reboot. Check system log to see anything happens.
Yes after scan, I needed to wait for a while then it unfroze. But I don't understand why I had multiple entries with processing status (the files are sequentially processed, so I don't see why I had that). After reboot and rescan, it seems to work better. I also did a pull of the last version. Was there any fixes between yesterday morning and today that can explain this?
Since ollama now supports multiple queries and batching, why don't we add an option to set maximum parallel files processing as an environment variable and as a service parameter then use that to speedup processing of files.
Also I suggest adding a progress value to the status of each file since we know already how many chunks to process, we can estimate a progress value. We can also add the number of entities and the number of connections and at the bottom the total number of entities and connections.
Finally, we need to implement the graph data recovery for at least the default version (the json and graphml) so that the Knowledge graph visualization can be viewed.
How to use this interface? I don't see any documentation, is it with fastapi? Thanks
@ArnoChenFx developed this new version of the interface. To access it you just install lightrag with [api] option, then to view it, you just go to your service address (example: http:localhost:9621/webui).
I suggest removing the /webui part as this would make it easier to access?
If that's OK with you I can do that now.
@ArnoChenFx did a great job. It is very iuntuitive. I made some friends test it and I didn't have to tell them much.
Although a progress bar is a nice feature, I don't consider it a priority for the project. Given that indexing large numbers of files can take hours, it's unlikely that users will stay on the page just to watch the progress. Instead, there are several more valuable features to consider adding:
- Include a failure list in the view, allowing users to identify and resolve issues before re-uploading.
- Display file names in the list view, so users can easily see which files have been successfully processed and which have failed.
- Enable multiple file and folder uploads, providing users with the ability to preview the file list and remove files before initiating the indexing process.
- Implement a delete feature, allowing users to make corrections without the need to reindex all files, which can be a significant task.
my ui is freezing until llm has responses, cannot switch to other tab pages or make any other api calls is it caused by using zhipu? I check the code and wonder if it really supports async calls
my ui is freezing until llm has responses, cannot switch to other tab pages or make any other api calls is it caused by using zhipu? I check the code and wonder if it really supports async calls
Based on my understanding, the LLM call operates asynchronously, whereas the large PDF(with image) content extraction using Docling does not. However, the latest version has eliminated Docling. Could you please provide a screenshot of the backend logs during the UI freeze?
yes - no more docling in latest version. Here is the screenshot
backend gets stuck at Upserting process, and once llm returns extracted info, everything backs to work.
I change to use OpenAI alike api, this problem get solved. I think problem is zhipu_complete_if_cache use
response = client.chat.completions.create(model=model, messages=messages, **kwargs)
not
client.chat.asyncCompletions.create
but asyncCompletions has some problems with stream response, so need a fix on it
oh, I found out the problem. We are using the classic logger which is storing the logs to disk and my tmp folder is full so this is causing a fail: logger.info(f"Inserting {len(data)} vectors to {self.namespace}") with No more space on the disk.
I would use ASCIIColors instead of the classic logger as it handles this kind of things or by default deactivate logging to files.
If the UI is freezing, there is no need to restart the server. Please refresh the webpage to reset the UI state. This will not affect any backend tasks.
- Now the failure list will be displayed. If you have failed tasks but they are not shown as "failed," it is likely that the document status in the database was not updated in time.
- The filename is not stored in the database, so the frontend cannot display it.
- The multi-file upload is an already implemented feature.
I've looked at zhipu's implementation code, and it's currently not using an asynchronous approach to call the LLM. Since zhipu's API is compatible with OpenAI, could we remove zhipu's implementation code and switch to using the OpenAI-alike mode to call zhipu's LLM? @zhouy13
May be we should add file name to database as an reference. @ArnoChenFx @YanSte
And zhipu_complete_if_cache is not handling stream response correctly. @zhouy13
Agree to add filename. Also need to consider the warning at the beginning of this PR #792.
When frontend summit a file for enqueuing, we should return a document id for future status inquiry. @YanSte @ArnoChenFx
@ArnoChenFx I know the multifiles import is implemented. I was thinking parallel processing of multiploe files and make the workers number programmable.
@ParisNeo @danielaskdd Could you please test the last version? The speed is improved.