modernization

[cloneluke]

upgrading to airflow 2.2.2-python3.8 package adding more apt packages for ease of use upgrade pip inline install aws cli for ease of use remove use of non-maintained pygresql in favor of more common/more maintained psycopg2 changed airflow entries to match airflow 2.x new syntax upgraded to cdk v2 upgraded all imports to the new cdk v2 structure upgraded rds to T3 instance class from T2 upgrading postgres rds engine to VER_13_4 upgraded typescript to 4.0.5 changing version naming standard to match airflow version + python version loosened strict typscript compiler to allow for dynamic variables fixed some deprecated cdk calls/references added ssh into fargate option

[cloneluke]

@thechaithanya @zevaverbach @realvz

[cloneluke]

@toricls

[toricls]

Hi @cloneluke, thank you for your contribution! Introducing Airflow v2.x is a really exciting idea I think!

To get this PR merged, I think we need some decisions from the authors (@realvz and @thechaithanya) because this repository is, AFAIK, deeply tied to our blog post here - "Running Airflow on AWS Fargate | Containers". So just introducing Airflow v2 here could break the relations between this repo and the blog post.

So one idea in my mind is:

• Having the changes in a new branch called like airflow-v2, not mainline
• Adding a note on mainline README like If you want to try this repo with Airflow v2, use this branch for example, and vice versa in airflow-v2 branch README

But again we still need some decisions (and also some tests) by the authors.

@realvz and @thechaithanya,

Could you give us your thoughts on this PR and my idea above? Of course different ideas are also very welcomed if you have ‪any 🤗‬

[thechaithanya]

Hi @toricls, I agree with the different branch approach and that's what exactly on my mind.

I'll post my comments on the CR changes later. On a side note, I don't have write permissions on this repo/package.

[toricls]

Great, thanks @thechaithanya!

I just added you as a collaborator in this repo (this is wired in the first place from my view because you're one of the original author 😉), so let me know if the permissions are not enough or anything else!

[thechaithanya]

Great, thanks @thechaithanya!

I just added you as a collaborator in this repo (this is wired in the first place from my view because you're one of the original author 😉), so let me know if the permissions are not enough or anything else!

Thanks. I haven't configured 2FA earlier. So, my permissions were revoked!!

[cloneluke]

Hi @cloneluke, thank you for your contribution! Introducing Airflow v2.x is a really exciting idea I think!

To get this PR merged, I think we need some decisions from the authors (@realvz and @thechaithanya) because this repository is, AFAIK, deeply tied to our blog post here - "Running Airflow on AWS Fargate | Containers". So just introducing Airflow v2 here could break the relations between this repo and the blog post.

So one idea in my mind is:

• Having the changes in a new branch called like airflow-v2, not mainline
• Adding a note on mainline README like If you want to try this repo with Airflow v2, use this branch for example, and vice versa in airflow-v2 branch README

But again we still need some decisions (and also some tests) by the authors.

@realvz and @thechaithanya,

Could you give us your thoughts on this PR and my idea above? Of course different ideas are also very welcomed if you have ‪any 🤗‬

Also maybe we should use tagging instead of branching for airflow major releases?

[feliperandopiercloud]

Will this PR be merged?