Jed Cunningham comments

Results 77 comments of


                                            Jed Cunningham

gitSync: add envFrom to support proxy settings from k8s secret

Thanks @rasulkarimov! Sorry for the slow merge. Congrats on your first commit 🎉

Fixed issue of new dag getting old dataset events.

cc @uranusjr

fix: delete pod when base container completes

Did you test `on_finish_action="keep_pod"`? We don't always delete the pods. I doubt we can do away with our istio handling like this, and I suspect that got broken somehow recently.

fix: delete pod when base container completes

I did a little digging, and that scenario was never covered when the feature was added in #33306 😞. I don't think we want to leave these pods still running.

fix: delete pod when base container completes

Yeah, it's tough to 1) do it generally 2) not delete pods and 3) not leave the pods running. That's why istio has (or tried to have it turns out)...

Improve scheduler loop by reducing repetative `TI.are_dependencies_met`

I worry a bit that this might have some unintended consequences. Would a simpler solution to this be adding a "blocked_by_upstream: bool" on TI, have the scheduler ignore those if...

Improve scheduler loop by reducing repetative `TI.are_dependencies_met`

> keep the data of blocked_by_upstream TIs for dagruns as an in-memory object? if there are not too many places where the TI changes its state The challenge is the...

Improve scheduler loop by reducing repetative `TI.are_dependencies_met`

> The problem with this method is that the scheduler would still include the dagrun and then check all the task instances to see whether they are blocked, but we...

Simple auth manager

An easy place to put the config (in the meantime?) would be in the webserver_config. I imagine that will live on regardless of whether FAB is used or not.

Reduce log noise from K8s Pod Operator for XCom

You could argue we shouldn't print them ever - they (xcom and task logs) are separate permissions after all.