Package causing Google Cloud Composer environment update to time out

[emilaleksanteri]

Question

I dont't know if its neccesarily this library; however, as soon as I added cloud-sql-python-connector to my PyPI packages on google cloud composer, building the composer environment times out when installing packages after 35 minutes. Without cloud-sql-python-connector the build takes ~5 minutes. Is there a reason why this packagage specifically causes the build to timeout even when specifying a version for it?

Code

No response

Additional Details

No response

[jackwotherspoon]

Hi @emilaleksanteri thanks for raising an issue on the Cloud SQL Python Connector 😄

This definitely seems like odd behavior as our package does not have many dependencies, and most are very commonly used packages.

https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/blob/a3486591d6f4f911b964fd2254ccf9a5ff80da20/setup.py#L27-L33

Maybe there is some type of dependency conflict with one of our dependencies and the Composer version you are using.

Which version of Cloud Composer are you using and seeing this behavior with?

Will take a look into this for you.

[emilaleksanteri]

Hi @emilaleksanteri thanks for raising an issue on the Cloud SQL Python Connector 😄

This definitely seems like odd behavior as our package does not have many dependencies, and most are very commonly used packages.

https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/blob/a3486591d6f4f911b964fd2254ccf9a5ff80da20/setup.py#L27-L33

Maybe there is some type of dependency conflict with one of our dependencies and the Composer version you are using.

Which version of Cloud Composer are you using and seeing this behavior with?

Will take a look into this for you.

Thank you for the swift response! Current composer setup is on this composer-2.9.4-airflow-2.9.3 and trying to add cloud-sql-python-connector==1.12.1 to the PyPI.

[emilaleksanteri]

Hi @emilaleksanteri thanks for raising an issue on the Cloud SQL Python Connector 😄 This definitely seems like odd behavior as our package does not have many dependencies, and most are very commonly used packages. https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/blob/a3486591d6f4f911b964fd2254ccf9a5ff80da20/setup.py#L27-L33

Maybe there is some type of dependency conflict with one of our dependencies and the Composer version you are using. Which version of Cloud Composer are you using and seeing this behavior with? Will take a look into this for you.

Thank you for the swift response! Current composer setup is on this composer-2.9.4-airflow-2.9.3 and trying to add cloud-sql-python-connector==1.12.1 to the PyPI.

@jackwotherspoon found that version 1.6.0 is the latest version that works with Composer airflow

[jackwotherspoon]

@emilaleksanteri This seems like a duplicate issue of https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/issues/1082 where the cryptography package version is causing the timeout as we force cryptography>=42.0.0 as of 1.7.0 of the Cloud SQL Python Connector (which is why 1.6.0 works for you).

Although I am surprised you are seeing this issue with composer-2.9.4-airflow-2.9.3 as I recently got the cryptography package bumped to cryptography==43.0.1 in newer Composer versions.

Do you mind giving the latest Composer version (composer-2.9.5-airflow-2.9.3) a try to see if the problem still persists?

[emilaleksanteri]

@jackwotherspoon Unfortionately the latest version I'm able to upgrade to is composer-2.9.4-airflow-2.9.3 but once a newer version becomes available I'll be sure to test and let you know!

[jackwotherspoon]

Hi @emilaleksanteri, just wanted to follow up and see if you have had a change to try any of the latest few composer versions? (we are now on composer-2.9.8-airflow-2.9.3 as the latest)

Hoping one of these newer versions fixes the dependency conflict and allows you to take advantage of the latest Cloud SQL Python Connector version 😄

[jackwotherspoon]

I am going to close this out as the latest composer versions should support the latest versions of the Cloud SQL Python Connector without issue.

If someone runs into this issue again feel free to re-open 😄