Kaxil Naik comments

Results 165 comments of


                                            Kaxil Naik

Add HTTP retry handling into task sdk's api.client

Thanks @jscheffl

SPIKE: Investigate what should be shared between server & client and determine if a common package is needed

> And I do not think we need a spike. For me it's just "common sense" (pun intended). Determining what is shared is as important and part of the spike...

SPIKE: Investigate what should be shared between server & client and determine if a common package is needed

>I think you misunderstood my description I think only task-sdk should be client and it should be "all" you need to run client. Even if it means duplication of some...

SPIKE: Investigate what should be shared between server & client and determine if a common package is needed

> One of the problems we are going to have is "triggerer" and "dag processor". Because effectively both of them have to have both "core" parts and "task-sdk" parts installed....

SPIKE: Investigate what should be shared between server & client and determine if a common package is needed

>I think we should have completely separate Executor Base distribution - and executor distributions should depend on it as well as on providers/task.sdk. I think this also can be achieve...

SPIKE: Investigate what should be shared between server & client and determine if a common package is needed

> Yeah - but you can't solve the knot "partially" if you do not know how you are going to solve https://github.com/apache/airflow/issues/51552. It is part of the same parent GH...

SPIKE: Investigate what should be shared between server & client and determine if a common package is needed

Yup, no disagreement on that >Sure. But I am telling that splitting dag-processor and triggerer out of the common is necessary and you need to know how your target split...

Start porting DAG definition code to the Task SDK

Merging this, one of the failures around k8s test that took longer than an hour looks like a transient one: ``` Progress: airflow-python-3.9-v1.28.13 Error during running tests for Python 3.9,...

Cope with Pycharm's oddity in detecting namespace packages

> @shahar1 @kaxil Could one of you test if this fixes it please? Yup, works now

Airflow 2 to 3 auto migration rules

We should add something for the [public API change](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html) too. API v1 won't work anymore. Those are being changed as part of AIP-84 to a new FastApi based app. GitHub...