ApeRAG icon indicating copy to clipboard operation
ApeRAG copied to clipboard

[Improvement] Use OpenDAL to abstract underlying storage for knowledge base files

Open iziang opened this issue 8 months ago • 1 comments

Description:
The Aperag knowledge base needs to support document synchronization from multiple sources (file uploads, emails, Feishu/Lark docs, URLs, etc.). These documents should be stored in Aperag for subsequent processing.

We propose using OpenDAL to abstract the underlying storage implementation. This will:
• Support multiple storage backends (object storage, local filesystem, etc.)
• Enable seamless switching between storage solutions based on user environment
• Maintain consistent operations regardless of storage backend

Key requirements:

  1. Implement OpenDAL as the storage abstraction layer
  2. Maintain existing synchronization channels
  3. Ensure backward compatibility

This abstraction will particularly benefit:
• Cloud deployments (compatible with S3/OSS/COS etc.)
• Hybrid environments (mixed cloud/on-premise)
• Local development setups

iziang avatar Apr 17 '25 01:04 iziang

This issue has been marked as stale because it has been open for 30 days with no activity

github-actions[bot] avatar May 19 '25 00:05 github-actions[bot]