Nathan Hadfield
Nathan Hadfield
Yes, that's true and presumably gets trickier if there are multiple levels of TaskGroups? I know there is a desire to make TaskGroups more than just a UI construct so...
Similar to your first idea, render the transfer operator as a cluster containing the from and to nodes?
@turbaszek I am currently testing Airflow v2.0.0b3 against the same DAGS we currently run on production against 1.10.12 and I can confirm that this is still an issue. Combined with...
To add some further context, I can consistently replicate this error on 2.0.0b3 on a very simple environment running two Docker containers - webserver and postgres - on a Python...
Not sure if this is relevant but, when the task was rescheduled five minutes later, I saw this. ``` [2020-12-03 11:57:07,266] {scheduler_job.py:1237} ERROR - Executor reports task instance finished (success)...
@turbaszek I just tried it again and I **couldn't** replicate this error again on 2.0.
Could this be something to do with updates to some of the `google-api` libraries between Airflow 2.8.4 and 2.9.1? ### 2.8.4 ``` google-api-core==2.17.1 google-api-python-client==2.122.0 google-auth==2.28.2 ``` ### 2.9.1 ``` google-api-core==2.19.0...
@potiuk Yes, I guess my confusion is more about why hook is giving the error in this context rather than the operator; especially as it doesn't seen to be caused...
@pankajastro Interesting. Downgrading does work but I'm curious to know what issue is and why the hook method gives the error but the operator does not.
I see. Thanks for the info.