databend icon indicating copy to clipboard operation
databend copied to clipboard

feat: Add azblob stage support

Open Xuanwo opened this issue 1 year ago โ€ข 2 comments

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR adds azblob stage support.

  • Fixes https://github.com/datafuselabs/databend/issues/15232

Tests

  • [x] Unit Test
  • [x] Logic Test
  • [ ] Benchmark Test
  • [ ] No Test - Explain why

Type of change

  • [ ] Bug Fix (non-breaking change which fixes an issue)
  • [x] New Feature (non-breaking change which adds functionality)
  • [ ] Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • [ ] Documentation Update
  • [ ] Refactoring
  • [ ] Performance Improvement
  • [ ] Other (please describe):

This change isโ€‚Reviewable

Xuanwo avatar Apr 15 '24 02:04 Xuanwo

PR Summary

  • Added support for Azure Blob Storage in storage configuration Enriched the storage configuration options by adding Azure Blob Storage support. This is implemented through the addition of Storage::Azblob in the general storage configuration and a related StorageAzblobConfig in the protobuf configuration implementation.

  • Added test configuration and cases for Azure Blob storage integration To ensure correct function and alignment with other storage options, added a new test suite specifically for Azure Blob Storage configurations. This includes a newly introduced test configuration v087_azblob_config and a dedicated test case test_decode_v87_azblob_config.

  • Modified location binding for Azure Blob Storage integration As part of adapting the Azure Blob Storage integration, we have revamped certain functions for URI location parsing (parse_uri_location) and param parsing (parse_azblob_params), enhancing how parameters are interpreted and ensuring consistent behaviour.

  • Updated SQL statements for Stage creation and deletion for Azure Blob Storage Added the relevant SQL DDL statements to create or replace and drop stages for Azure Blob Storage configurations in our testing. This assures that the operations performed on this storage option are aligned with the usual operations for other storage types.

what-the-diff[bot] avatar Apr 15 '24 02:04 what-the-diff[bot]

It seems this PR breaking something, many tests failed.

bohutang avatar Apr 15 '24 13:04 bohutang