pathway icon indicating copy to clipboard operation
pathway copied to clipboard

Support Azure Blob Storage as a persistence backend

Open zxqfd555 opened this issue 7 months ago • 0 comments

Is your feature request related to a problem? Please describe. Currently, supported persistence backend are local file system and S3. Support for Azure Blob Storage is also required.

Describe the solution you'd like The implementation of Azure persistence backend can be done in the following steps:

  • Implement the key-value storage interface for Azure. Since Rust is used as a framework's core language, a Rust crate must be used to provide access to Azure. azure_storage_blobs can be used, as one of the most popular options;
  • Support Azure Blob Storage in the DataStorage config and add an option to the persistent storage config.
  • Add Azure as a persistence backend option in the Python frontend.
  • Finally, add it as an option to the persistence integration tests.

The integration is expected to be non-breaking, as it introduces a new backend via an existing interface.

zxqfd555 avatar May 25 '25 19:05 zxqfd555