infiniflow
[Feature Request]: File Management and Knowledge Base
Is there an existing issue for the same feature request?
- [X] I have checked the existing issues.
Is your feature request related to a problem?
Knowledge Base functionality, duplicate parsing of the same document in different knowledge bases from File Management
Describe the feature you'd like
The way I see it, File Management uploads and parses the files, Knowledge Base links to the files inside File Management via tags, which reduces duplicate parsing of files, and calls Knowledge Base when called.
Describe implementation you've considered
No response
Documentation, adoption, use case
No response
Additional information
No response
I don't get it. This function has already full filled.
Uploading files through File Management and then associating them to multiple different knowledge bases requires each knowledge base to be parsed separately, which can reduce the number of times the files are parsed if they are parsed in file management and then associated.
@KevinHuSh how can we add single document to multiple knowledge bases and avoid duplication of file on the storage and chunks in the Elasticsearch? This is the question.
Clear. Duplication in storage will not happen. But, duplicated chunks in ES, I'm afraid there're for now.