dremio-oss
dremio-oss copied to clipboard
Google Cloud Storage whitelist buckets by regex
Google Cloud Storage whitelist buckets by regex
Description
This pull request adds a new feature to Google Cloud Storage plugin that allows for the whitelisting of buckets using regular expressions. This feature can be particularly useful in a data mesh context, where it's common to segregate data domains and products based on specific naming conventions.
Before this change, whitelisting buckets had to be done manually, which was a time-consuming and error-prone task. With this new feature, users can now choose to whitelist buckets by listing their names individually or by defining a regular expression to match multiple buckets simultaneously. This makes the process of whitelisting buckets more efficient and less prone to errors, which is especially important in a data mesh context where data quality and governance are critical.
Examples
Here are some examples of how this new feature could be used:
- Whitelist all buckets that contain the word "sales":
^sales-.*
- Whitelist all buckets that belong to a specific domain:
^mydomain-.*
- Whitelist all buckets that start with a specific date:
^2023-03-.*
Screenshots
The screenshot below shows the classic option:
This one shows the new option with the regex:
.