Update dependency urllib3 to v2 [SECURITY]
This PR contains the following updates:
| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| urllib3 (changelog) | ==1.26.15 -> ==2.5.0 |
GitHub Vulnerability Alerts
CVE-2023-43804
urllib3 doesn't treat the Cookie HTTP header special or provide any helpers for managing cookies over HTTP, that is the responsibility of the user. However, it is possible for a user to specify a Cookie header and unknowingly leak information via HTTP redirects to a different origin if that user doesn't disable redirects explicitly.
Users must handle redirects themselves instead of relying on urllib3's automatic redirects to achieve safe processing of the Cookie header, thus we decided to strip the header by default in order to further protect users who aren't using the correct approach.
Affected usages
We believe the number of usages affected by this advisory is low. It requires all of the following to be true to be exploited:
- Using an affected version of urllib3 (patched in v1.26.17 and v2.0.6)
- Using the
Cookieheader on requests, which is mostly typical for impersonating a browser. - Not disabling HTTP redirects
- Either not using HTTPS or for the origin server to redirect to a malicious origin.
Remediation
- Upgrading to at least urllib3 v1.26.17 or v2.0.6
- Disabling HTTP redirects using
redirects=Falsewhen sending requests. - Not using the
Cookieheader.
CVE-2023-45803
urllib3 previously wouldn't remove the HTTP request body when an HTTP redirect response using status 303 "See Other" after the request had its method changed from one that could accept a request body (like POST) to GET as is required by HTTP RFCs. Although the behavior of removing the request body is not specified in the section for redirects, it can be inferred by piecing together information from different sections and we have observed the behavior in other major HTTP client implementations like curl and web browsers.
From RFC 9110 Section 9.3.1:
A client SHOULD NOT generate content in a GET request unless it is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported.
Affected usages
Because the vulnerability requires a previously trusted service to become compromised in order to have an impact on confidentiality we believe the exploitability of this vulnerability is low. Additionally, many users aren't putting sensitive data in HTTP request bodies, if this is the case then this vulnerability isn't exploitable.
Both of the following conditions must be true to be affected by this vulnerability:
- If you're using urllib3 and submitting sensitive information in the HTTP request body (such as form data or JSON)
- The origin service is compromised and starts redirecting using 303 to a malicious peer or the redirected-to service becomes compromised.
Remediation
You can remediate this vulnerability with any of the following steps:
- Upgrade to a patched version of urllib3 (v1.26.18 or v2.0.7)
- Disable redirects for services that you aren't expecting to respond with redirects with
redirects=False. - Disable automatic redirects with
redirects=Falseand handle 303 redirects manually by stripping the HTTP request body.
CVE-2024-37891
When using urllib3's proxy support with ProxyManager, the Proxy-Authorization header is only sent to the configured proxy, as expected.
However, when sending HTTP requests without using urllib3's proxy support, it's possible to accidentally configure the Proxy-Authorization header even though it won't have any effect as the request is not using a forwarding proxy or a tunneling proxy. In those cases, urllib3 doesn't treat the Proxy-Authorization HTTP header as one carrying authentication material and thus doesn't strip the header on cross-origin redirects.
Because this is a highly unlikely scenario, we believe the severity of this vulnerability is low for almost all users. Out of an abundance of caution urllib3 will automatically strip the Proxy-Authorization header during cross-origin redirects to avoid the small chance that users are doing this on accident.
Users should use urllib3's proxy support or disable automatic redirects to achieve safe processing of the Proxy-Authorization header, but we still decided to strip the header by default in order to further protect users who aren't using the correct approach.
Affected usages
We believe the number of usages affected by this advisory is low. It requires all of the following to be true to be exploited:
- Setting the
Proxy-Authorizationheader without using urllib3's built-in proxy support. - Not disabling HTTP redirects.
- Either not using an HTTPS origin server or for the proxy or target origin to redirect to a malicious origin.
Remediation
- Using the
Proxy-Authorizationheader with urllib3'sProxyManager. - Disabling HTTP redirects using
redirects=Falsewhen sending requests. - Not using the
Proxy-Authorizationheader.
CVE-2025-50181
urllib3 handles redirects and retries using the same mechanism, which is controlled by the Retry object. The most common way to disable redirects is at the request level, as follows:
resp = urllib3.request("GET", "https://httpbin.org/redirect/1", redirect=False)
print(resp.status)
# 302
However, it is also possible to disable redirects, for all requests, by instantiating a PoolManager and specifying retries in a way that disable redirects:
import urllib3
http = urllib3.PoolManager(retries=0) # should raise MaxRetryError on redirect
http = urllib3.PoolManager(retries=urllib3.Retry(redirect=0)) # equivalent to the above
http = urllib3.PoolManager(retries=False) # should return the first response
resp = http.request("GET", "https://httpbin.org/redirect/1")
However, the retries parameter is currently ignored, which means all the above examples don't disable redirects.
Affected usages
Passing retries on PoolManager instantiation to disable redirects or restrict their number.
By default, requests and botocore users are not affected.
Impact
Redirects are often used to exploit SSRF vulnerabilities. An application attempting to mitigate SSRF or open redirect vulnerabilities by disabling redirects at the PoolManager level will remain vulnerable.
Remediation
You can remediate this vulnerability with the following steps:
- Upgrade to a patched version of urllib3. If your organization would benefit from the continued support of urllib3 1.x, please contact [email protected] to discuss sponsorship or contribution opportunities.
- Disable redirects at the
request()level instead of thePoolManager()level.
Release Notes
urllib3/urllib3 (urllib3)
v2.5.0
==================
Features
- Added support for the
compression.zstdmodule that is new in Python 3.14. SeePEP 784 <https://peps.python.org/pep-0784/>_ for more information. (#​3610 <https://github.com/urllib3/urllib3/issues/3610>__) - Added support for version 0.5 of
hatch-vcs(#​3612 <https://github.com/urllib3/urllib3/issues/3612>__)
Bugfixes
- Fixed a security issue where restricting the maximum number of followed
redirects at the
urllib3.PoolManagerlevel via theretriesparameter did not work. - Made the Node.js runtime respect redirect parameters such as
retriesandredirects. - Raised exception for
HTTPResponse.shutdownon a connection already released to the pool. (#​3581 <https://github.com/urllib3/urllib3/issues/3581>__) - Fixed incorrect
CONNECTstatement when using an IPv6 proxy withconnection_from_host. Previously would not be wrapped in[]. (#​3615 <https://github.com/urllib3/urllib3/issues/3615>__)
v2.4.0
==================
Features
- Applied PEP 639 by specifying the license fields in pyproject.toml. (
#​3522 <https://github.com/urllib3/urllib3/issues/3522>__) - Updated exceptions to save and restore more properties during the pickle/serialization process. (
#​3567 <https://github.com/urllib3/urllib3/issues/3567>__) - Added
verify_flagsoption tocreate_urllib3_contextwith a default ofVERIFY_X509_PARTIAL_CHAINandVERIFY_X509_STRICTfor Python 3.13+. (#​3571 <https://github.com/urllib3/urllib3/issues/3571>__)
Bugfixes
- Fixed a bug with partial reads of streaming data in Emscripten. (
#​3555 <https://github.com/urllib3/urllib3/issues/3555>__)
Misc
- Switched to uv for installing development dependecies. (
#​3550 <https://github.com/urllib3/urllib3/issues/3550>__) - Removed the
multiple.intoto.jsonlasset from GitHub releases. Attestation of release files since v2.3.0 can be found on PyPI. (#​3566 <https://github.com/urllib3/urllib3/issues/3566>__)
v2.3.0
==================
Features
- Added
HTTPResponse.shutdown()to stop any ongoing or future reads for a specific response. It callsshutdown(SHUT_RD)on the underlying socket. This feature wassponsored by LaunchDarkly <https://opencollective.com/urllib3/contributions/815307>. (#​2868 <https://github.com/urllib3/urllib3/issues/2868>) - Added support for JavaScript Promise Integration on Emscripten. This enables more efficient WebAssembly
requests and streaming, and makes it possible to use in Node.js if you launch it as
node --experimental-wasm-stack-switching. (#​3400 <https://github.com/urllib3/urllib3/issues/3400>__) - Added the
proxy_is_tunnelingproperty toHTTPConnectionandHTTPSConnection. (#​3285 <https://github.com/urllib3/urllib3/issues/3285>__) - Added pickling support to
NewConnectionErrorandNameResolutionError. (#​3480 <https://github.com/urllib3/urllib3/issues/3480>__)
Bugfixes
- Fixed an issue in debug logs where the HTTP version was rendering as "HTTP/11" instead of "HTTP/1.1". (
#​3489 <https://github.com/urllib3/urllib3/issues/3489>__)
Deprecations and Removals
- Removed support for Python 3.8. (
#​3492 <https://github.com/urllib3/urllib3/issues/3492>__)
v2.2.3
==================
Features
- Added support for Python 3.13. (
#​3473 <https://github.com/urllib3/urllib3/issues/3473>__)
Bugfixes
- Fixed the default encoding of chunked request bodies to be UTF-8 instead of ISO-8859-1.
All other methods of supplying a request body already use UTF-8 starting in urllib3 v2.0. (
#​3053 <https://github.com/urllib3/urllib3/issues/3053>__) - Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting https://github.com/python/cpython/issues/103472. (
#​3252 <https://github.com/urllib3/urllib3/issues/3252>__) - Adjust tolerance for floating-point comparison on Windows to avoid flakiness in CI (
#​3413 <https://github.com/urllib3/urllib3/issues/3413>__) - Fixed a crash where certain standard library hash functions were absent in restricted environments. (
#​3432 <https://github.com/urllib3/urllib3/issues/3432>__) - Fixed mypy error when adding to
HTTPConnection.default_socket_options. (#​3448 <https://github.com/urllib3/urllib3/issues/3448>__)
HTTP/2 (experimental)
HTTP/2 support is still in early development.
-
Excluded Transfer-Encoding: chunked from HTTP/2 request body (
#​3425 <https://github.com/urllib3/urllib3/issues/3425>__) -
Added version checking for
h2(https://pypi.org/project/h2/) usage.Now only accepting supported h2 major version 4.x.x. (
#​3290 <https://github.com/urllib3/urllib3/issues/3290>__) -
Added a probing mechanism for determining whether a given target origin supports HTTP/2 via ALPN. (
#​3301 <https://github.com/urllib3/urllib3/issues/3301>__) -
Add support for sending a request body with HTTP/2 (
#​3302 <https://github.com/urllib3/urllib3/issues/3302>__)
Deprecations and Removals
- Note for downstream distributors: the
_version.pyfile has been removed and is now created at build time by hatch-vcs. (#​3412 <https://github.com/urllib3/urllib3/issues/3412>__) - Drop support for end-of-life PyPy3.8 and PyPy3.9. (
#​3475 <https://github.com/urllib3/urllib3/issues/3475>__)
v2.2.2
==================
- Added the
Proxy-Authorizationheader to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set viaRetry.remove_headers_on_redirect. - Allowed passing negative integers as
amtto read methods ofhttp.client.HTTPResponseas an alternative toNone. (#​3122 <https://github.com/urllib3/urllib3/issues/3122>__) - Fixed return types representing copying actions to use
typing.Self. (#​3363 <https://github.com/urllib3/urllib3/issues/3363>__)
v2.2.1
==================
- Fixed issue where
InsecureRequestWarningwas emitted for HTTPS connections when using Emscripten. (#​3331 <https://github.com/urllib3/urllib3/issues/3331>__) - Fixed
HTTPConnectionPool.urlopento stop automatically casting non-proxy headers toHTTPHeaderDict. This change was premature as it did not apply to proxy headers andHTTPHeaderDictdoes not handle byte header values correctly yet. (#​3343 <https://github.com/urllib3/urllib3/issues/3343>__) - Changed
InvalidChunkLengthtoProtocolErrorwhen response terminates before the chunk length is sent. (#​2860 <https://github.com/urllib3/urllib3/issues/2860>__) - Changed
ProtocolErrorto be more verbose on incomplete reads with excess content. (#​3261 <https://github.com/urllib3/urllib3/issues/3261>__)
v2.2.0
==================
- Added support for
Emscripten and Pyodide <https://urllib3.readthedocs.io/en/latest/reference/contrib/emscripten.html>, including streaming support in cross-origin isolated browser environments where threading is enabled. (#​2951 <https://github.com/urllib3/urllib3/issues/2951>) - Added support for
HTTPResponse.read1()method. (#​3186 <https://github.com/urllib3/urllib3/issues/3186>__) - Added rudimentary support for HTTP/2. (
#​3284 <https://github.com/urllib3/urllib3/issues/3284>__) - Fixed issue where requests against urls with trailing dots were failing due to SSL errors
when using proxy. (
#​2244 <https://github.com/urllib3/urllib3/issues/2244>__) - Fixed
HTTPConnection.proxy_is_verifiedandHTTPSConnection.proxy_is_verifiedto be always set to a boolean after connecting to a proxy. It could beNonein some cases previously. (#​3130 <https://github.com/urllib3/urllib3/issues/3130>__) - Fixed an issue where
headerspassed in a request withjson=would be mutated (#​3203 <https://github.com/urllib3/urllib3/issues/3203>__) - Fixed
HTTPSConnection.is_verifiedto be set toFalsewhen connecting from a HTTPS proxy to an HTTP target. It was set toTruepreviously. (#​3267 <https://github.com/urllib3/urllib3/issues/3267>__) - Fixed handling of new error message from OpenSSL 3.2.0 when configuring an HTTP proxy as HTTPS (
#​3268 <https://github.com/urllib3/urllib3/issues/3268>__) - Fixed TLS 1.3 post-handshake auth when the server certificate validation is disabled (
#​3325 <https://github.com/urllib3/urllib3/issues/3325>__) - Note for downstream distributors: To run integration tests, you now need to run the tests a second
time with the
--integrationpytest flag. (#​3181 <https://github.com/urllib3/urllib3/issues/3181>__)
v2.1.0
==================
- Removed support for the deprecated urllib3[secure] extra. (
#​2680 <https://github.com/urllib3/urllib3/issues/2680>__) - Removed support for the deprecated SecureTransport TLS implementation. (
#​2681 <https://github.com/urllib3/urllib3/issues/2681>__) - Removed support for the end-of-life Python 3.7. (
#​3143 <https://github.com/urllib3/urllib3/issues/3143>__) - Allowed loading CA certificates from memory for proxies. (
#​3065 <https://github.com/urllib3/urllib3/issues/3065>__) - Fixed decoding Gzip-encoded responses which specified
x-gzipcontent-encoding. (#​3174 <https://github.com/urllib3/urllib3/issues/3174>__)
v2.0.7
==================
- Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other" redirect responses.
v2.0.6
==================
- Added the
Cookieheader to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set viaRetry.remove_headers_on_redirect.
v2.0.5
==================
- Allowed pyOpenSSL third-party module without any deprecation warning. (
#​3126 <https://github.com/urllib3/urllib3/issues/3126>__) - Fixed default
blocksizeofHTTPConnectionclasses to match high-level classes. Previously was 8KiB, now 16KiB. (#​3066 <https://github.com/urllib3/urllib3/issues/3066>__)
v2.0.4
==================
- Added support for union operators to
HTTPHeaderDict(#​2254 <https://github.com/urllib3/urllib3/issues/2254>__) - Added
BaseHTTPResponsetourllib3.__all__(#​3078 <https://github.com/urllib3/urllib3/issues/3078>__) - Fixed
urllib3.connection.HTTPConnectionto raise thehttp.client.connectaudit event to have the same behavior as the standard library HTTP client (#​2757 <https://github.com/urllib3/urllib3/issues/2757>__) - Relied on the standard library for checking hostnames in supported PyPy releases (
#​3087 <https://github.com/urllib3/urllib3/issues/3087>__)
v2.0.3
==================
- Allowed alternative SSL libraries such as LibreSSL, while still issuing a warning as we cannot help users facing issues with implementations other than OpenSSL. (
#​3020 <https://github.com/urllib3/urllib3/issues/3020>__) - Deprecated URLs which don't have an explicit scheme (
#​2950 <https://github.com/urllib3/urllib3/pull/2950>_) - Fixed response decoding with Zstandard when compressed data is made of several frames. (
#​3008 <https://github.com/urllib3/urllib3/issues/3008>__) - Fixed
assert_hostname=Falseto correctly skip hostname check. (#​3051 <https://github.com/urllib3/urllib3/issues/3051>__)
v2.0.2
==================
- Fixed
HTTPResponse.stream()to continue yielding bytes if buffered decompressed data was still available to be read even if the underlying socket is closed. This prevents a compressed response from being truncated. (#​3009 <https://github.com/urllib3/urllib3/issues/3009>__)
v2.0.1
==================
- Fixed a socket leak when fingerprint or hostname verifications fail. (
#​2991 <https://github.com/urllib3/urllib3/issues/2991>__) - Fixed an error when
HTTPResponse.read(0)was the firstreadcall or when the internal response body buffer was otherwise empty. (#​2998 <https://github.com/urllib3/urllib3/issues/2998>__)
v2.0.0
==================
Read the v2.0 migration guide <https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html>__ for help upgrading to the latest version of urllib3.
Removed
- Removed support for Python 2.7, 3.5, and 3.6 (
#​883 <https://github.com/urllib3/urllib3/issues/883>,#​2336 <https://github.com/urllib3/urllib3/issues/2336>). - Removed fallback on certificate
commonNameinmatch_hostname()function. This behavior was deprecated in May 2000 in RFC 2818. Instead onlysubjectAltNameis used to verify the hostname by default. To enable verifying the hostname againstcommonNameuseSSLContext.hostname_checks_common_name = True(#​2113 <https://github.com/urllib3/urllib3/issues/2113>__). - Removed support for Python with an
sslmodule compiled with LibreSSL, CiscoSSL, wolfSSL, and all other OpenSSL alternatives. Python is moving to require OpenSSL with PEP 644 (#​2168 <https://github.com/urllib3/urllib3/issues/2168>__). - Removed support for OpenSSL versions earlier than 1.1.1 or that don't have SNI support.
When an incompatible OpenSSL version is detected an
ImportErroris raised (#​2168 <https://github.com/urllib3/urllib3/issues/2168>__). - Removed the list of default ciphers for OpenSSL 1.1.1+ and SecureTransport as their own defaults are already secure (
#​2082 <https://github.com/urllib3/urllib3/issues/2082>__). - Removed
urllib3.contrib.appengine.AppEngineManagerand support for Google App Engine Standard Environment (#​2044 <https://github.com/urllib3/urllib3/issues/2044>__). - Removed deprecated
Retryoptionsmethod_whitelist,DEFAULT_REDIRECT_HEADERS_BLACKLIST(#​2086 <https://github.com/urllib3/urllib3/issues/2086>__). - Removed
urllib3.HTTPResponse.from_httplib(#​2648 <https://github.com/urllib3/urllib3/issues/2648>__). - Removed default value of
Nonefor therequest_contextparameter ofurllib3.PoolManager.connection_from_pool_key. This change should have no effect on users as the default value ofNonewas an invalid option and was never used (#​1897 <https://github.com/urllib3/urllib3/issues/1897>__). - Removed the
urllib3.requestmodule.urllib3.request.RequestMethodshas been made a private API. This change was made to ensure thatfrom urllib3 import requestimported the top-levelrequest()function instead of theurllib3.requestmodule (#​2269 <https://github.com/urllib3/urllib3/issues/2269>__). - Removed support for SSLv3.0 from the
urllib3.contrib.pyopenssleven when support is available from the compiled OpenSSL library (#​2233 <https://github.com/urllib3/urllib3/issues/2233>__). - Removed the deprecated
urllib3.contrib.ntlmpoolmodule (#​2339 <https://github.com/urllib3/urllib3/issues/2339>__). - Removed
DEFAULT_CIPHERS,HAS_SNI,USE_DEFAULT_SSLCONTEXT_CIPHERS, from the private moduleurllib3.util.ssl_(#​2168 <https://github.com/urllib3/urllib3/issues/2168>__). - Removed
urllib3.exceptions.SNIMissingWarning(#​2168 <https://github.com/urllib3/urllib3/issues/2168>__). - Removed the
_prepare_connmethod fromHTTPConnectionPool. Previously this was only used to callHTTPSConnection.set_cert()byHTTPSConnectionPool(#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). - Removed
tls_in_tls_requiredproperty fromHTTPSConnection. This is now determined from theschemeparameter inHTTPConnection.set_tunnel()(#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). - Removed the
strictparameter/attribute fromHTTPConnection,HTTPSConnection,HTTPConnectionPool,HTTPSConnectionPool, andHTTPResponse(#​2064 <https://github.com/urllib3/urllib3/issues/2064>__).
Deprecated
- Deprecated
HTTPResponse.getheaders()andHTTPResponse.getheader()which will be removed in urllib3 v2.1.0. Instead useHTTPResponse.headersandHTTPResponse.headers.get(name, default). (#​1543 <https://github.com/urllib3/urllib3/issues/1543>,#​2814 <https://github.com/urllib3/urllib3/issues/2814>). - Deprecated
urllib3.contrib.pyopensslmodule which will be removed in urllib3 v2.1.0 (#​2691 <https://github.com/urllib3/urllib3/issues/2691>__). - Deprecated
urllib3.contrib.securetransportmodule which will be removed in urllib3 v2.1.0 (#​2692 <https://github.com/urllib3/urllib3/issues/2692>__). - Deprecated
ssl_versionoption in favor ofssl_minimum_version.ssl_versionwill be removed in urllib3 v2.1.0 (#​2110 <https://github.com/urllib3/urllib3/issues/2110>__). - Deprecated the
strictparameter ofPoolManager.connection_from_context()as it's not longer needed in Python 3.x. It will be removed in urllib3 v2.1.0 (#​2267 <https://github.com/urllib3/urllib3/issues/2267>__) - Deprecated the
NewConnectionError.poolattribute which will be removed in urllib3 v2.1.0 (#​2271 <https://github.com/urllib3/urllib3/issues/2271>__). - Deprecated
format_header_param_html5andformat_header_paramin favor offormat_multipart_header_param(#​2257 <https://github.com/urllib3/urllib3/issues/2257>__). - Deprecated
RequestField.header_formatterparameter which will be removed in urllib3 v2.1.0 (#​2257 <https://github.com/urllib3/urllib3/issues/2257>__). - Deprecated
HTTPSConnection.set_cert()method. Instead pass parameters to theHTTPSConnectionconstructor (#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). - Deprecated
HTTPConnection.request_chunked()method which will be removed in urllib3 v2.1.0. Instead passchunked=TruetoHTTPConnection.request()(#​1985 <https://github.com/urllib3/urllib3/issues/1985>__).
Added
- Added top-level
urllib3.requestfunction which uses a preconfigured module-globalPoolManagerinstance (#​2150 <https://github.com/urllib3/urllib3/issues/2150>__). - Added the
jsonparameter tourllib3.request(),PoolManager.request(), andConnectionPool.request()methods to send JSON bodies in requests. Using this parameter will set the headerContent-Type: application/jsonifContent-Typeisn't already defined. Added support for parsing JSON response bodies withHTTPResponse.json()method (#​2243 <https://github.com/urllib3/urllib3/issues/2243>__). - Added type hints to the
urllib3module (#​1897 <https://github.com/urllib3/urllib3/issues/1897>__). - Added
ssl_minimum_versionandssl_maximum_versionoptions which setSSLContext.minimum_versionandSSLContext.maximum_version(#​2110 <https://github.com/urllib3/urllib3/issues/2110>__). - Added support for Zstandard (RFC 8878) when
zstandard1.18.0 or later is installed. Added thezstdextra which installs thezstandardpackage (#​1992 <https://github.com/urllib3/urllib3/issues/1992>__). - Added
urllib3.response.BaseHTTPResponseclass. All future response classes will be subclasses ofBaseHTTPResponse(#​2083 <https://github.com/urllib3/urllib3/issues/2083>__). - Added
FullPoolErrorwhich is raised whenPoolManager(block=True)and a connection is returned to a full pool (#​2197 <https://github.com/urllib3/urllib3/issues/2197>__). - Added
HTTPHeaderDictto the top-levelurllib3namespace (#​2216 <https://github.com/urllib3/urllib3/issues/2216>__). - Added support for configuring header merging behavior with HTTPHeaderDict
When using a
HTTPHeaderDictto provide headers for a request, by default duplicate header values will be repeated. But ifcombine=Trueis passed into a call toHTTPHeaderDict.add, then the added header value will be merged in with an existing value into a comma-separated list (X-My-Header: foo, bar) (#​2242 <https://github.com/urllib3/urllib3/issues/2242>__). - Added
NameResolutionErrorexception when a DNS error occurs (#​2305 <https://github.com/urllib3/urllib3/issues/2305>__). - Added
proxy_assert_hostnameandproxy_assert_fingerprintkwargs toProxyManager(#​2409 <https://github.com/urllib3/urllib3/issues/2409>__). - Added a configurable
backoff_maxparameter to theRetryclass. If a custombackoff_maxis provided to theRetryclass, it will replace theRetry.DEFAULT_BACKOFF_MAX(#​2494 <https://github.com/urllib3/urllib3/issues/2494>__). - Added the
authorityproperty to the Url class as per RFC 3986 3.2. This property should be used in place ofnetlocfor users who want to include the userinfo (auth) component of the URI (#​2520 <https://github.com/urllib3/urllib3/issues/2520>__). - Added the
schemeparameter toHTTPConnection.set_tunnelto configure the scheme of the origin being tunnelled to (#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). - Added the
is_closed,is_connectedandhas_connected_to_proxyproperties toHTTPConnection(#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). - Added optional
backoff_jitterparameter toRetry. (#​2952 <https://github.com/urllib3/urllib3/issues/2952>__)
Changed
-
Changed
urllib3.response.HTTPResponse.readto respect the semantics ofio.BufferedIOBaseregardless of compression. Specifically, this method:- Only returns an empty bytes object to indicate EOF (that is, the response has been fully consumed).
- Never returns more bytes than requested.
- Can issue any number of system calls: zero, one or multiple.
If you want each
urllib3.response.HTTPResponse.readcall to issue a single system call, you need to disable decompression by settingdecode_content=False(#​2128 <https://github.com/urllib3/urllib3/issues/2128>__). -
Changed
urllib3.HTTPConnection.getresponseto return an instance ofurllib3.HTTPResponseinstead ofhttp.client.HTTPResponse(#​2648 <https://github.com/urllib3/urllib3/issues/2648>__). -
Changed
ssl_versionto instead set the correspondingSSLContext.minimum_versionandSSLContext.maximum_versionvalues. Regardless ofssl_versionpassedSSLContextobjects are now constructed usingssl.PROTOCOL_TLS_CLIENT(#​2110 <https://github.com/urllib3/urllib3/issues/2110>__). -
Changed default
SSLContext.minimum_versionto beTLSVersion.TLSv1_2in line with Python 3.10 (#​2373 <https://github.com/urllib3/urllib3/issues/2373>__). -
Changed
ProxyErrorto wrap any connection error (timeout, TLS, DNS) that occurs when connecting to the proxy (#​2482 <https://github.com/urllib3/urllib3/pull/2482>__). -
Changed
urllib3.util.create_urllib3_contextto not override the system cipher suites with a default value. The new default will be cipher suites configured by the operating system (#​2168 <https://github.com/urllib3/urllib3/issues/2168>__). -
Changed
multipart/form-dataheader parameter formatting matches the WHATWG HTML Standard as of 2021-06-10. Control characters in filenames are no longer percent encoded (#​2257 <https://github.com/urllib3/urllib3/issues/2257>__). -
Changed the error raised when connecting via HTTPS when the
sslmodule isn't available fromSSLErrortoImportError(#​2589 <https://github.com/urllib3/urllib3/issues/2589>__). -
Changed
HTTPConnection.request()to always use lowercase chunk boundaries when sending requests withTransfer-Encoding: chunked(#​2515 <https://github.com/urllib3/urllib3/issues/2515>__). -
Changed
enforce_content_lengthdefault to True, preventing silent data loss when reading streamed responses (#​2514 <https://github.com/urllib3/urllib3/issues/2514>__). -
Changed internal implementation of
HTTPHeaderDictto usedictinstead ofcollections.OrderedDictfor better performance (#​2080 <https://github.com/urllib3/urllib3/issues/2080>__). -
Changed the
urllib3.contrib.pyopensslmodule to wrapOpenSSL.SSL.Errorwithssl.SSLErrorinPyOpenSSLContext.load_cert_chain(#​2628 <https://github.com/urllib3/urllib3/issues/2628>__). -
Changed usage of the deprecated
socket.errortoOSError(#​2120 <https://github.com/urllib3/urllib3/issues/2120>__). -
Changed all parameters in the
HTTPConnectionandHTTPSConnectionconstructors to be keyword-only excepthostandport(#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). -
Changed
HTTPConnection.getresponse()to set the socket timeout fromHTTPConnection.timeoutvalue before reading data from the socket. This previously was done manually by theHTTPConnectionPoolcallingHTTPConnection.sock.settimeout(...)(#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). -
Changed the
_proxy_hostproperty to_tunnel_hostinHTTPConnectionPoolto more closely match how the property is used (value inHTTPConnection.set_tunnel()) (#​1985 <https://github.com/urllib3/urllib3/issues/1985>__). -
Changed name of
Retry.BACK0FF_MAXto beRetry.DEFAULT_BACKOFF_MAX. -
Changed TLS handshakes to use
SSLContext.check_hostnamewhen possible (#​2452 <https://github.com/urllib3/urllib3/pull/2452>__). -
Changed
server_hostnameto behave like other parameters only used byHTTPSConnectionPool(#​2537 <https://github.com/urllib3/urllib3/pull/2537>__). -
Changed the default
blocksizeto 16KB to match OpenSSL's default read amounts (#​2348 <https://github.com/urllib3/urllib3/pull/2348>__). -
Changed
HTTPResponse.read()to raise an error when calling withdecode_content=Falseafter usingdecode_content=Trueto prevent data loss (#​2800 <https://github.com/urllib3/urllib3/issues/2800>__).
Fixed
- Fixed thread-safety issue where accessing a
PoolManagerwith many distinct origins would cause connection pools to be closed while requests are in progress (#​1252 <https://github.com/urllib3/urllib3/issues/1252>__). - Fixed an issue where an
HTTPConnectioninstance would erroneously reuse the socket read timeout value from reading the previous response instead of a newly configured connect timeout. Instead now ifHTTPConnection.timeoutis updated before sending the next request the new timeout value will be used (#​2645 <https://github.com/urllib3/urllib3/issues/2645>__). - Fixed
socket.error.errnowhen raised from pyOpenSSL'sOpenSSL.SSL.SysCallError(#​2118 <https://github.com/urllib3/urllib3/issues/2118>__). - Fixed the default value of
HTTPSConnection.socket_optionsto matchHTTPConnection(#​2213 <https://github.com/urllib3/urllib3/issues/2213>__). - Fixed a bug where
headerswould be modified by theremove_headers_on_redirectfeature (#​2272 <https://github.com/urllib3/urllib3/issues/2272>__). - Fixed a reference cycle bug in
urllib3.util.connection.create_connection()(#​2277 <https://github.com/urllib3/urllib3/issues/2277>__). - Fixed a socket leak if
HTTPConnection.connect()fails (#​2571 <https://github.com/urllib3/urllib3/pull/2571>__). - Fixed
urllib3.contrib.pyopenssl.WrappedSocketandurllib3.contrib.securetransport.WrappedSocketclose methods (#​2970 <https://github.com/urllib3/urllib3/issues/2970>__)
v1.26.20
====================
- Fixed a crash where certain standard library hash functions were absent in
FIPS-compliant environments.
(
#​3432 <https://github.com/urllib3/urllib3/issues/3432>__) - Replaced deprecated dash-separated setuptools entries in
setup.cfg. (#​3461 <https://github.com/urllib3/urllib3/pull/3461>__) - Took into account macOS setting
ECONNRESETinstead ofEPROTOTYPEin its newer versions. (#​3416 <https://github.com/urllib3/urllib3/pull/3416>__) - Backported changes to our tests and CI configuration from v2.x to support
testing with CPython 3.12 and 3.13.
(
#​3436 <https://github.com/urllib3/urllib3/pull/3436>__)
v1.26.19
====================
- Added the
Proxy-Authorizationheader to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set viaRetry.remove_headers_on_redirect. - Fixed handling of OpenSSL 3.2.0 new error message for misconfiguring an HTTP proxy as HTTPS. (
#​3405 <https://github.com/urllib3/urllib3/issues/3405>__)
v1.26.18
====================
- Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other" redirect responses.
v1.26.17
====================
- Added the
Cookieheader to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set viaRetry.remove_headers_on_redirect. (#​3139 <https://github.com/urllib3/urllib3/pull/3139>_)
v1.26.16
====================
- Fixed thread-safety issue where accessing a
PoolManagerwith many distinct origins would cause connection pools to be closed while requests are in progress (#​2954 <https://github.com/urllib3/urllib3/pull/2954>_)
Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
- [ ] If you want to rebase/retry this PR, check this box
This PR was generated by Mend Renovate. View the repository job log.