bigquery-utils
bigquery-utils copied to clipboard
Feature Request: Authorized View Generator / Refresher script
Often customers have a source table and an authorized view which drops a few blacklisted fields. Sometimes the schema of the underlying table changes and the view definition drops fields it shouldn't.
Let's create a script that will pull the schema from an existing table and drop a list of blacklisted fields (or sub-fields) and generate the SQL for a view.
The script should take the following inputs:
- source table (fully qualified project.dataset.table)
- destination view ( fully qualified project.dataset.view)
- list of fields to drop
Considerations:
- Handle dropping fields from within structs in a nested schema.
- Implementing this in Golang would be nice so it can be a compiled CLI utility and for ease of usage from terraform modules.
Nice to Have:
- Include automation to Create the view and reinstate it's authorization on the source dataset.
Ugly Python Prototype: gist