dbt-core
dbt-core copied to clipboard
[Feature] enable on-run-end hook and associated contexts to be made available via `dbt clone`
Is this your first time submitting a feature request?
- [X] I have read the expectations for open source contributors
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
Hi Team,
Requesting to have dbt clone
trigger on-run-end hooks, from my understanding and testing it currently doesn't: https://docs.getdbt.com/reference/project-configs/on-run-start-on-run-end
In our case, we have custom grants applied via on-run-end
hooks that also leverages the database_schemas
context.
This doesn't currently work for dbt clone
, and affects the some of our slice builds which expect data to be cloned into a slice database.schema
for use externally such as in notebooks & Sagemaker.
We've worked around this for the moment via running a modified version of the on-run-end
hook as an operation, but would be nice to see this built in like other dbt invocations.
Let me know if you've any questions!
Describe alternatives you've considered
As mentioned, we've got a workaround working where the macro can be called as an operation as opposed to an on-run-end
hook. This doesn't afford as access to the database_schemas
context however.
Who will this benefit?
Any usecase where you want to clone production data and have on-run-end
hooks run without also having to do a dbt run/build/snapshot
etc
Are you interested in contributing this feature?
I'm not sure I'm technical enough to assist with contributions but happy to sense check
Anything else?
snippet of current workaround:
{%- if feature_branch(branch_name) and target.name.split('_')[0] not in ["local", "dev"] -%}
{%- for (database, schema) in resolved_database_schemas -%}
{%- for role in roles %}
{# setting all the grants to be run #}
{% set database_schema_grants %}
grant all on schema {{ database }}.{{ schema }} to role {{ role }};
grant all on all tables in schema {{ database }}.{{ schema }} to role {{ role }};
grant all on all views in schema {{ database }}.{{ schema }} to role {{ role }};
grant all on future tables in schema {{ database }}.{{ schema }} to role {{ role }};
grant all on future views in schema {{ database }}.{{ schema }} to role {{ role }};
{% endset %}
{# on-run-end hooks automatically call the compiled SQL directly, but operations do not. #}
{% if flags.WHICH != 'run-operation' %}
{{ database_schema_grants }}
{% else %}
{% do run_query(database_schema_grants) %}
{% endif %}
{%- endfor -%}
{%- endfor -%}
{%- endif -%}