bigquery-utils icon indicating copy to clipboard operation
bigquery-utils copied to clipboard

Feature Request: Authorized View Generator / Refresher script

Open jaketf opened this issue 6 years ago • 0 comments

Often customers have a source table and an authorized view which drops a few blacklisted fields. Sometimes the schema of the underlying table changes and the view definition drops fields it shouldn't.

Let's create a script that will pull the schema from an existing table and drop a list of blacklisted fields (or sub-fields) and generate the SQL for a view.

The script should take the following inputs:

  • source table (fully qualified project.dataset.table)
  • destination view ( fully qualified project.dataset.view)
  • list of fields to drop

Considerations:

  • Handle dropping fields from within structs in a nested schema.
  • Implementing this in Golang would be nice so it can be a compiled CLI utility and for ease of usage from terraform modules.

Nice to Have:

  • Include automation to Create the view and reinstate it's authorization on the source dataset.

Ugly Python Prototype: gist

jaketf avatar Aug 28 '19 22:08 jaketf