dbt-athena
dbt-athena copied to clipboard
Zero-downtime for table recalculation within dbt run
Hi I'm sorry if I'm missing something - I'm new in dbt and just trying to switch the project from custom data model management to dbt
Does dbt-athena supports table recalculation without downtime while it's being recalculated?
What workaround I used to do in my own script: to give every CTAS an unique name (i.e. select * from <name>_<random>
- so once new version of ctas is calculated I switch a view to the new version and drop the old one. All usages reference only view and the whole thing works without downtime
Will appreciate your thoughts and comments!
I had a shot at implementing a PoC for this feature as well, maybe that could help! My code is at https://github.com/iasoon/dbt-athena/commit/cca66da328058e40ec5490cdb7a5a4589b042440. It seems to work in simple tests, but it's not battle-tested :)