dbt-snow-mask icon indicating copy to clipboard operation
dbt-snow-mask copied to clipboard

Model build time unapply

Open robscriva opened this issue 3 years ago • 1 comments

Currently for the unapply operation in the apply_masking_policy_list_for_models macro it loops through the nodes in the graph. Perhaps this could be decoupled to allow a pre-hook unapply to the model as well.

So there would be two approaches to applying policies:

  • unapply / apply in pre-post hook steps
  • unapply / apply in batch via run-operations or on-run-start/end

robscriva avatar Sep 13 '21 04:09 robscriva

@robscriva , Sorry it took so long to get back to you on this one

  • Just was testing this out and unapply works in post/pre-hooks
  • Here are the steps I followed, added post-hook
  post-hook: 
    - "{{ dbt_snow_mask.unapply_masking_policy('models') }}"
  • Executed the models by dbt run and it ran fine removing the policy
00:17:54  Running with dbt=1.0.1
00:17:54  Unable to do partial parsing because a project config has changed
00:18:00  Found 1 model, 0 tests, 0 snapshots, 0 analyses, 404 macros, 0 operations, 1 seed file, 1 source, 0 exposures, 0 metrics
00:18:00  
00:18:04  Concurrency: 10 threads (target='dev')
00:18:04  
00:18:04  1 of 1 START view model staging.stg_customer.................................... [RUN]
00:18:05  00:18:05 | creating masking policy           : DEMO_DB.STAGING.MP_ENCRYPT_PII
00:18:06  00:18:06 | unapplying masking policy to model  : DEMO_DB.STAGING.MP_ENCRYPT_PII on DEMO_DB.staging.stg_customer.email
00:18:07  1 of 1 OK created view model staging.stg_customer............................... [SUCCESS 1 in 2.52s]
00:18:07  
00:18:07  Finished running 1 view model in 7.09s.
00:18:07  
00:18:07  Completed successfully
00:18:07  
00:18:07  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
  • Did I miss something in understanding your request ?

entechlog avatar Apr 23 '22 00:04 entechlog