Tatiana Al-Chueyr
Tatiana Al-Chueyr
I believe the issue Rodrigo is facing is likely due to https://github.com/astronomer/astronomer-cosmos/blob/3c9cf6fbceb369efcb6854731833558e8b487749/cosmos/operators/virtualenv.py#L72 This will only be executed in case the operator execution succeeds: https://github.com/astronomer/astronomer-cosmos/blob/3c9cf6fbceb369efcb6854731833558e8b487749/cosmos/operators/virtualenv.py#L106-L107 We should change this to use...
Relates to #927
Follow up with the community in #airflow-dbt: https://apache-airflow.slack.com/archives/C059CC42E9W/p1696339581151849 https://apache-airflow.slack.com/archives/C059CC42E9W/p1696364828572859 https://apache-airflow.slack.com/archives/C059CC42E9W/p1696498945508569
The work was started (and is unstable) in the branch `set-inlets-during-runtime`. Requires further work.
Origin: #305 Airflow related issue: https://github.com/apache/airflow/issues/34206
After a few discussions with Airflow core developers over the past months, including with @uranusjr, he came up with the proposal for Dataset Aliases, would would allow us to set...
@linchun3 this is an excellent proposal - including the suggestion by @dwreeves. We'd love to have this feature! Please, feel free to work on this!
This issue led an Astronomer customer to face an incident, due to an excessive amount of logs related to orphaning datasets: https://astronomer.zendesk.com/agent/tickets/63128
@roadan @vargacypher @alexisvannier @CorsettiS @genzgd @epikhinm @Gaploid Since this work is still relevant, and the PR got stale, @pankajastro is rebasing the original implementation in #1016, so we can get...
@vargacypher, thanks a lot; we just merged #1016 and let you know once we have an alpha with this feature!