[Improvement] Use OpenDAL to abstract underlying storage for knowledge base files
Description:
The Aperag knowledge base needs to support document synchronization from multiple sources (file uploads, emails, Feishu/Lark docs, URLs, etc.). These documents should be stored in Aperag for subsequent processing.
We propose using OpenDAL to abstract the underlying storage implementation. This will:
• Support multiple storage backends (object storage, local filesystem, etc.)
• Enable seamless switching between storage solutions based on user environment
• Maintain consistent operations regardless of storage backend
Key requirements:
- Implement OpenDAL as the storage abstraction layer
- Maintain existing synchronization channels
- Ensure backward compatibility
This abstraction will particularly benefit:
• Cloud deployments (compatible with S3/OSS/COS etc.)
• Hybrid environments (mixed cloud/on-premise)
• Local development setups
This issue has been marked as stale because it has been open for 30 days with no activity