dremio-oss icon indicating copy to clipboard operation
dremio-oss copied to clipboard

Google Cloud Storage whitelist buckets by regex

Open damiano1996 opened this issue 1 year ago • 1 comments

Google Cloud Storage whitelist buckets by regex

Description

This pull request adds a new feature to Google Cloud Storage plugin that allows for the whitelisting of buckets using regular expressions. This feature can be particularly useful in a data mesh context, where it's common to segregate data domains and products based on specific naming conventions.

Before this change, whitelisting buckets had to be done manually, which was a time-consuming and error-prone task. With this new feature, users can now choose to whitelist buckets by listing their names individually or by defining a regular expression to match multiple buckets simultaneously. This makes the process of whitelisting buckets more efficient and less prone to errors, which is especially important in a data mesh context where data quality and governance are critical.

Examples

Here are some examples of how this new feature could be used:

  • Whitelist all buckets that contain the word "sales": ^sales-.*
  • Whitelist all buckets that belong to a specific domain: ^mydomain-.*
  • Whitelist all buckets that start with a specific date: ^2023-03-.*

Screenshots

The screenshot below shows the classic option:

dremio_01_list

This one shows the new option with the regex:

dremio_02_regex.

damiano1996 avatar Mar 25 '23 22:03 damiano1996