clickhouse-operator
clickhouse-operator copied to clipboard
Add ClickhouseBackup Plugin Support
Overview
This PR introduces a new Backup plugin architecture to the ClickHouse Operator, enabling automated backup operations via an external plugin service. Issue: #1795
Current PR Includes
1. Operator Enhancements
-
CRDs
- ClickhouseBackup (CHB) — manage one-time backup requests with S3 configuration, whitelist/blacklist filtering, and method/plugin settings.
-
Controller Updates
- Operator reconciles CHB/CHSB CRs and communicates with a gRPC backup plugin using JSON-serialized request data.
- Adds plugin discovery via CR labels/annotations.
- Implements lifecycle management and status reporting.
2. Go Plugin Skeleton
- Introduces a gRPC interface for backup operations using
BackupRequestandBackupResultmessages. - Plugin architecture allows future extension for operations like restore, encryption, incremental backups, etc.
Motivation & Benefits
- Modular Design: Backup behavior is decoupled from the operator — update, replace, or extend the plugin without touching core operator logic.
- Extensibility: Support for diverse backup implementations (e.g., S3, local disk, incremental, etc.).
- Production-Ready: Enables defined workflows for both one-off and scheduled backups, ensuring reliable defense against data loss.
TODO
- Restore Workflow: Implement a corresponding restore service via plugin.
- Test Cases: Implement the test cases
- ClickhouseScheduledBackup (CHSB) Implement code for scheduled backup
- Generic gRPC Client: Implement a reusable gRPC client inside the operator for communication with different plugins.
Testing Notes
- Validate CHB and CHSB workflows manually using a mock plugin server.
- Status updates should reflect backup success or failure states.
- Scheduling logic (cron parsing, suspension, etc.) requires validation.
Will this get accepted.
Would be really cool to see this getting merged