Support urllib3 >= 2.0.0

[danigm]

The urllib3.request.RequestMethods has been moved to urllib3._request_methods.RequestMethods.

This patch changes the usage to continue working with the latest release, but now it's a "private" class in a "private" module, so maybe it's worth to start to think about updating this code to use the public API.

https://urllib3.readthedocs.io/en/stable/changelog.html#removed

See https://github.com/googleapis/google-auth-library-python/pull/1283

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[danigm]

The tests are working with this change, but it looks like it also work if instead of using directly the RequestMethods class we use some of the classes that inherit from it, like HTTPConnectionPool: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connectionpool.html#urllib3.HTTPConnectionPool

[clundin25]

Thank you for the contribution! It will take some time to accept this patch. There are downstream dependencies that depend on this functionality.

[pquentin]

Hello, urllib3 maintainer here 👋

It's unfortunate that urllib3 has always documented its internals as if they were a public API, because it makes any changes difficult.

Can you please clarify what downstream dependencies depend on exactly?

[clundin25]

Hi @pquentin,

We use PyOpenSSL to implement non-file based credentials w/ mTLS. We are investigating other approaches but until we have a solution in place we cannot migrate.

Deprecation notice here: https://github.com/urllib3/urllib3/issues/2691

[pquentin]

OK, so RequestsMethod isn't really an issue, this is good to know. Others have reported that non-file TLS is the only the reason why they still use pyOpenSSL, and it's true that we don't have a good alternative today.

It's only deprecated, not removed, though. Is the issue the deprecation warning? I don't want to be pushy: this is your software. But I'm trying to better understand the blockers.

[clundin25]

@danigm Let me double check. My assumption is that we do not want the deprecation warning to propagate.

I appreciate the effort / time to update this for us!

[arithmetic1728]

RequestsMethod is one reason. Another reason (as @clundin25 mentioned) is that urllib3 will remove pyopenssl support in v2.1.0. For v2.0.0 it currently just raises a warning but that could bother the users, so we would like to keep urllib3 < 2.0 for now until we find a good way to handle both issues.

[mgorny]

Thanks for the patch. I'm going to take it into Gentoo to let us unblock urllib3-2.

[pquentin]

@arithmetic1728 @clundin25 I have discussed this with the other urllib3 maintainers and we're going to undeprecate pyOpenSSL. We won't add the deprecation and warning back unless there's a good solution for this problem (like a third-party library or a fix in CPython itself).

Would that be enough for you to upgrade?

[arithmetic1728]

@pquentin This is great news! Which release will pyopenssl support be back?

The only thing left to update urllib3 would be RequestsMethod, we might want let our AuthorizedHttp class inherit urllib3._request_methods.RequestMethods as in https://github.com/googleapis/google-auth-library-python/pull/1290/files. Is urllib3 planning to change urllib3._request_methods.RequestMethods in the future?

[sethmlarson]

@arithmetic1728 Happy to hear you're willing to migrate! :partying_face: I opened an issue to remove the deprecation warning for pyOpenSSL.

The RequestMethods class is private, since it's really just a convenience class for us to not need to reimplement all the logic around .request() for all the different layers (PoolManager, ConnectionPool, ProxyManager). You could copy all the relevant code into AuthorizedHttp instead of subclassing? It's mostly shuffling headers and request bodies around.

[arithmetic1728]

@sethmlarson Thank you for the suggestions. We will upgrade urllib3 once the depreciation message is removed.

[pquentin]

Hello! https://pypi.org/project/urllib3/2.0.5/ is now out without the deprecation warning.

[arithmetic1728]

@pquentin awesome. Thank you!

[antoni-szych-rtbhouse]

Seems like this was partially done in https://github.com/googleapis/google-auth-library-python/pull/1389, but without fixing the docstrings. Maybe @danigm can rebase this PR to master in order to fix the outdated docstrings?