lightningstream
lightningstream copied to clipboard
Adding Azure blob support
This pull request adds support for Azure Blob Storage as a backend for storing snapshots in Lightning Stream, updates dependencies, and introduces new Prometheus metrics for improved observability of snapshot operations. It also updates documentation and the development environment to reflect these changes.
Note this depends on the Simpleblob PR to be merged and released with 0.4.0 tag.
Azure Blob Storage Integration
- Added support for Azure Blob Storage as a storage backend by registering the Azure backend in
main.goand updating theStoragetype inconfig.goto include"azure". [1] [2] - Updated
docker-compose.ymlto include an Azurite service for local Azure Blob Storage emulation and example configuration for Azure backend indocker/pdns/lightningstream.yaml. [1] [2]
Metrics Enhancements
- Introduced new Prometheus metrics for tracking snapshot age, timestamp, generation ID, and download times in both the syncer and receiver components, improving monitoring and troubleshooting capabilities. [1] [2] [3] [4] [5] [6] [7] [8] [9]
Configuration and Documentation Updates
- Expanded the
README.mdwith detailed instructions for configuring Azure Blob Storage, including static credentials and service principal authentication, as well as an options reference table and example usage with Azurite.
Dependency Updates
- Upgraded
github.com/PowerDNS/simpleblobto v0.4.0 and several other dependencies to their latest versions ingo.modto support Azure Blob Storage and improve reliability. [1] [2]
Logging Improvements
- Changed log level for new snapshot detection in the receiver from debug to info for better visibility. [1] [2]