pallets
Some relative redirect strings incorrectly bypass iri_to_uri normalization.
Summary
It appears logic introduced in https://github.com/pallets/werkzeug/pull/2695 to work-around the itms-services: schema oddity has an overly broad assumption that is results in invalid URIs due to not percent-encoding relative IRIs as intended.
Ultimately, this can trick browsers into behaving badly when faced with invalid relative URL locations. But I think it is ours to fix in werkzeug because the current code is emitting an invalid location header per RFCs (and the presumed intention of the code.)
RFC 3986 for URIs has a section 2 that essentially says URI components should be percent-encoded if they aren't part of the "reserved" or explicitly "unreserved" lists of characters. As an case-study: backslashes should be percent-encoded. Indeed, werkzeug seems to intent to encode them, but this issue describes the case where the typical encoding logic is not being applied.
Replication
As an example consider an app expecting to use redirect(relative_uri) and expecting werkzeug to properly encode the Location: header (as it historically would before the mentioned work-around)
from werkzeug.utils import redirect
problem_relative_uri = '/\\\\github.com?extra=data&blank=#fragment'
response = redirect(problem_relative_uri)
response.location
# '/\\\\github.com?extra=data&blank=#fragment'
Note the backslashes (\) in this URI are /not/ being encoded as %5C .
As a smaller test case:
from werkzeug.urls import iri_to_uri, _invalid_iri_to_uri
problem_relative_uri = '/\\\\github.com?extra=data&blank=#fragment'
_invalid_iri_to_uri(problem_relative_uri)
# '/\\\\github.com?extra=data&blank=#fragment'
# :-(
Expected Behavior
The previous encoding behavior before the wrapper was desirable
iri_to_uri(problem_relative_uri)
# '/%5C%5Cgithub.com?extra=data&blank=#fragment'
# :-)
Environment:
- Python version: 3.12.1
- Werkzeug version: 3.0.1
Relevant Code
https://github.com/pallets/werkzeug/blob/d3dd65a27388fbd39d146caacf2563639ba622f0/src/werkzeug/wrappers/response.py#L482-L483
https://github.com/pallets/werkzeug/blob/d3dd65a27388fbd39d146caacf2563639ba622f0/src/werkzeug/urls.py#L167-L185
After @davidism 's prompt in the original issue to also file an upstream Python issue (https://github.com/python/cpython/issues/104139#issuecomment-1539176043), Python's urlib.parse was improved with the itms-services: url schema knowledge in Python 3.12.
The cpython comment thread also mentions a potentially more tightly scoped workaround for the "_invalid_iri" issue on older python versions:
if "itms-services" not in urllib.parse.uses_netloc:
urllib.parse.uses_netloc.append("itms-services")
Should we adopt this? Or otherwise rethink the current conditionals bypassing iri_to_uri(location_iri)?
Yes, I like that solution, since it allows users to tell Python in general about any special scheme they need. Note that you won't see response.location encoded in your example, that only happens at the last moment when converting the Response to HTTP.
