InvalidHeader exception when trying to add a ":method" header field (breaking change in 2.28.0)

[Alex8768765]

My app is making a POST request enforcing some specific header fields. Among them, I was able to explicitly enforce a ":method" header field. But since 2.28.0, this now raises the following error:

requests.exceptions.InvalidHeader: Invalid leading whitespace, reserved character(s), or returncharacter(s) in header name: ':method'

Enforcing this header field is possibly a widespread practice as this is part of the requirements for sending notifications through Apple's push notification servers.

[nateprewitt]

Hi @Alex8768765,

Can you please provide the info we originally requested in the issue template? The header you're using isn't valid in the HTTP spec, so it seems unlikely this would be widespread. Python itself rejects sending a header like this in our supported versions.

>>> requests.__version__
'2.27.1'
>>> r = requests.get('https://httpbin.org/get', headers={':method': 'test'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/http/client.py", line 1322, in _send_request
    self.putheader(hdr, value)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/site-packages/urllib3/connection.py", line 224, in putheader
    _HTTPConnection.putheader(self, header, *values)
  File "/Users/nateprewitt/.pyenv/versions/3.7.12/lib/python3.7/http/client.py", line 1249, in putheader
    raise ValueError('Invalid header name %r' % (header,))
ValueError: Invalid header name b':method'

[Alex8768765]

Apologies, I missed that the impacted piece of code was using Requests together with a 3rd party HTTP/2 adapter.

from hyper.contrib import HTTP20Adapter
import requests
s = requests.Session()
s.mount("https://httpbin.org/get", HTTP20Adapter())
r = s.post("https://httpbin.org/get", headers={':method': 'test'})

That 3rd party adapter does not seem to be a maintained project as we speak, so I understand my issue is essentially finding the proper way to make HTTP/2 requests.

[nateprewitt]

Hrmm, well that does pose a problem. While Requests only supports http/1.1, Hyper has been "compatible" for quite a while.

I'll need to take a look at possible escape hatches. Ideally, we can update the hyper example to still use requests and bypass the header checks, but I'm not sure how/if that's accomplished.

[nateprewitt]

After a cursory look, I don’t think we can escape this at the adapter level. The immediate fix is to use the PreparedRequest workflow in the Requests docs and set the headers after the Request is prepared. This can be handed to the Session and should work.

I’d like to see how often hyper is being used with Requests still before we look at merging in a code change to bypass this.

[abirabedinkhan]

Did you guys found any solutions

[abirabedinkhan]

Traceback (most recent call last):
  File "C:\Users\abira\Desktop\Bkash Marchent API\app.py", line 52, in <module>
    print(api.TableData())
  File "C:\Users\abira\Desktop\Bkash Marchent API\app.py", line 44, in TableData
    data = AUTH.post('https://cpp.bka.sh/merchant-portal-backend/merchant/dashboard', json={})
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 484, in prepare_request
    p.prepare(
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\models.py", line 369, in prepare
    self.prepare_headers(headers)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\models.py", line 491, in prepare_headers
    check_header_validity(header)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\utils.py", line 1042, in check_header_validity
    _validate_header_part(name, "name", HEADER_VALIDATORS[type(name)][0])
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\utils.py", line 1048, in _validate_header_part
    raise InvalidHeader(
requests.exceptions.InvalidHeader: Invalid leading whitespace, reserved character(s), or returncharacter(s) in header name: ':authority'

This is what shows in my error

[BlondinkaQ]

replace in header '\r' or '\t' or '\n' to empty symbol, this solved my problem

[afriedma]

replace in header '\r' or '\t' or '\n' to empty symbol, this solved my problem

Hi, what is the workaround for “:” I am adding ":authority" and the request fails

[wuzeyuuu]

Traceback (most recent call last):
  File "C:\Users\abira\Desktop\Bkash Marchent API\app.py", line 52, in <module>
    print(api.TableData())
  File "C:\Users\abira\Desktop\Bkash Marchent API\app.py", line 44, in TableData
    data = AUTH.post('https://cpp.bka.sh/merchant-portal-backend/merchant/dashboard', json={})
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 484, in prepare_request
    p.prepare(
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\models.py", line 369, in prepare
    self.prepare_headers(headers)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\models.py", line 491, in prepare_headers
    check_header_validity(header)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\utils.py", line 1042, in check_header_validity
    _validate_header_part(name, "name", HEADER_VALIDATORS[type(name)][0])
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\utils.py", line 1048, in _validate_header_part
    raise InvalidHeader(
requests.exceptions.InvalidHeader: Invalid leading whitespace, reserved character(s), or returncharacter(s) in header name: ':authority'

This is what shows in my error

I also met this question，here is my code from hyper.contrib import HTTP20Adapter

url = "https://xxxxxx.com/tag/xxxx/page/1.html" headers = {':authority': 'xxxxxx.com', ':method': 'GET', ':path': '/tag/xxxx/page/1', ':scheme': 'https', 'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8', 'Sec-Fetch-Mode':'navigate', 'Sec-Fetch-Site':'none', 'Sec-Fetch-User':'?1', 'Upgrade-Insecure-Requests':'1', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36'}

sessions = requests.session() sessions.mount('https://xxxx.com', HTTP20Adapter()) response = sessions.get(url = url,headers = headers)

And I get an error like: Invalid leading whitespace, reserved character(s), or returncharacter(s) in header name: ':authority' It made me crasy.

[ahmadkarimi4]

same is my problem too

headers={'Key:'+str(self.key) : 'Sign:'+str(sign)}

i wannna do this but same error pops up

[cloudy-sfu]

Same problem.

[wushenchao]

+1

[abhishekanish]

requests.exceptions.InvalidHeader: Invalid leading whitespace, reserved character(s), or returncharacter(s) in header name: ':authority'

+1

Any resolution found?

[lcoimbra]

+1

[Crypss22]

Traceback (most recent call last):
  File "C:\Users\abira\Desktop\Bkash Marchent API\app.py", line 52, in <module>
    print(api.TableData())
  File "C:\Users\abira\Desktop\Bkash Marchent API\app.py", line 44, in TableData
    data = AUTH.post('https://cpp.bka.sh/merchant-portal-backend/merchant/dashboard', json={})
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\sessions.py", line 484, in prepare_request
    p.prepare(
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\models.py", line 369, in prepare
    self.prepare_headers(headers)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\models.py", line 491, in prepare_headers
    check_header_validity(header)
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\utils.py", line 1042, in check_header_validity
    _validate_header_part(name, "name", HEADER_VALIDATORS[type(name)][0])
  File "C:\Users\abira\Desktop\Bkash Marchent API\env\lib\site-packages\requests\utils.py", line 1048, in _validate_header_part
    raise InvalidHeader(
requests.exceptions.InvalidHeader: Invalid leading whitespace, reserved character(s), or returncharacter(s) in header name: ':authority'

This is what shows in my error

I'm also receiving these and can't seem to find a solution.. Any updates?

[sigmavirus24]

For those arriving here, this is not a frequently encountered issue, but the solution is in Nate's comment which I will quote here:

After a cursory look, I don’t think we can escape this at the adapter level. The immediate fix is to use the PreparedRequest workflow in the Requests docs and set the headers after the Request is prepared. This can be handed to the Session and should work.

I’d like to see how often hyper is being used with Requests still before we look at merging in a code change to bypass this.

Please read this and the documentation and do not post comments with "+1" or asking for updates. Further comments like this will result in locking the discussion