Daniel Standish
Daniel Standish
> So the question is whether we can cut it now or not. It'd be possible for a user to swap in TaskHandlerWithCustomFormatter in airflow_local_settings, right? sorry didn't understand @jedcunningham...
i was more curious about "the only question is whether we can cut it now or not" i guess what you mean is, whether we can remove it immediately, or...
I think we probably could just remove it. It would be sorta not too bad if someone was actually using it because the cluster would no run I think so...
this should be obsoleted by this PR https://github.com/apache/airflow/pull/39336 i'm just working through bunch of test fixes cus we make a _lot_ of assertions about try_number because it's always been so...
> @dstandish I agree the linked PR is a good long term solution but this PR is good enough for a bugfix in 2.9.x series with 2.10 release few months...
Ok so the fix is, don't try to adopt completed pods. You mention that not big consequence if we don't do this because we can periodically delete completed pods in...
In case it is important to adopt pods, but we need a fix to do it safely, just throwing out some ideas before i disappear for holiday. The scheduler can...
> The completed pods are not a problem because they consume no resources, and they will be deleted during the airflow cleanup-pods cronjob execution. However, a TaskInstance can get stuck...
Nice findings. Looks promising. Thanks for that. Ok here's another scenario. The task OOMs and therefore cannot report its state by itself.
Are there any other open questions on this one?