feat: Add Aliyun SLS (Simple Log Service) integration for workflow execution logging

[adongfan]

Summary

This PR implements Aliyun Simple Log Service (SLS) integration as an alternative storage backend for workflow execution logs, addressing the challenge described in #27117 where PostgreSQL-based execution records consume significant database space and create stability/performance risks.

• Resolves #27117

Key Features:

1. Aliyun SLS LogStore Integration

• New extension module (ext_logstore) that initializes SLS projects, logstores, and indexes on application startup
• Automatic index schema synchronization from SQLAlchemy models, ensuring consistency between database schema and LogStore indexes
• Support for storing workflow execution and node execution logs in Aliyun SLS
• Idempotent initialization that safely handles existing projects and logstores

2. Dual-Write Support for Safe Migration

• Configurable dual-write mode (LOGSTORE_DUAL_WRITE_ENABLED) that writes to both SLS and SQL database simultaneously
• Enables safe migration from SQL-based logging to SLS without data loss

3. Repository Pattern Implementation

• Four new repository classes implementing the workflow execution repository interface:
  • LogstoreWorkflowExecutionRepository - for workflow runs
  • LogstoreWorkflowNodeExecutionRepository - for workflow node executions
  • LogstoreApiWorkflowRunRepository - for API workflow runs
  • LogstoreApiWorkflowNodeExecutionRepository - for API workflow node executions
• Automatic type mapping from SQLAlchemy column types to LogStore index types (text, long, double, json)
• Full compatibility with existing domain models (WorkflowExecution & WorkflowNodeExecution)

4. Production-Ready Configuration

• Environment variables for Aliyun SLS authentication and configuration
• Configurable TTL (Time-To-Live) for log retention with automatic cleanup
• Debug logging support when SQLALCHEMY_ECHO=true

Configuration Example:

# Enable Aliyun SLS LogStore
ALIYUN_SLS_ACCESS_KEY_ID=your_access_key_id
ALIYUN_SLS_ACCESS_KEY_SECRET=your_access_key_secret
ALIYUN_SLS_ENDPOINT=cn-hangzhou.log.aliyuncs.com
ALIYUN_SLS_REGION=cn-hangzhou
ALIYUN_SLS_PROJECT_NAME=dify-workflow-logs

# Optional: Configure TTL and dual-write behavior
ALIYUN_SLS_LOGSTORE_TTL=30  # days, default: 30
LOGSTORE_DUAL_WRITE_ENABLED=false  # Set to true during migration phase

Benefits:

1. Scalability: Purpose-built log storage handles high-throughput workflow executions efficiently
2. Cost Efficiency: Reduces load on primary PostgreSQL database, freeing resources for transactional data
3. Storage Optimization: Automatic TTL-based cleanup prevents unbounded growth
4. Analytics Ready: Native log analysis and visualization capabilities in Aliyun SLS console
5. Safe Migration: Dual-write support ensures zero data loss during transition period
6. Auto Schema Sync: Index schemas automatically sync with database migrations via SQLAlchemy introspection

Architecture Notes:

This implementation follows the repository pattern established in #23966 and aligns with the domain model architecture improvements referenced in #19850 and #19428. The LogStore repositories implement the same interfaces as the existing SQLAlchemy repositories, ensuring seamless integration with the workflow execution system.

Testing Notes:

• ✅ All backend linting checks passed (make lint)
• ✅ Type checking passed with BasedPyright (make type-check)
• ✅ Import contracts validated (8 contracts kept)
• ✅ Code formatted with Ruff
• Manual testing recommended with Aliyun SLS credentials

Screenshots

Not applicable - backend infrastructure change with no UI changes.

Checklist

• [ ] This change requires a documentation update, included: Dify Document
• [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• [x] I've updated the documentation accordingly.
• [x] I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[hieheihei]

Here’s the demo：

1. Configure Logstore storage in .env

2. Start the dify-api, and the ext_logstore plugin will create/check the Logstore resource and update the index during startup.

3.Run a workflow, and you can view the execution records on the Logs and Monitoring page (the record data comes from Logstore).

4.View the stored execution record data in the Alibaba Cloud SLS console.

5.View the read and write traffic from Dify in the "Service Logs" section.

[laipz8200]

almost LGTM, please make the linter happy :xD

[adongfan]

almost LGTM, please make the linter happy :xD

Thanks for the review! I’ve fixed the linter issues — should be good now.