WSGI header spoofing via underscore/dash conflation

[TomiBelan]

Gunicorn normalizes headers by converting dashes to underscores. Both Foo-Bar and Foo_Bar become HTTP_FOO_BAR. Additionally, if the HTTP request contains both Foo-Bar: a and Foo_Bar: b, Gunicorn merges them to {'HTTP_FOO_BAR': 'a,b'}.

This can lead to vulnerabilities in some web apps. If a header is used in a security-sensitive way (for instance, passing authentication information along from a front-end proxy), even if the proxy carefully strips any incoming value for X-Auth-User, an attacker may be able to provide an X-Auth_User header (with underscore) and bypass this protection.

Another example: if the attacker sets X_Forwarded_For: 5.6.7.8 (fake IP) and the front proxy sets X-Forwarded-For: 1.2.3.4 (real IP), the Python app could see 1.2.3.4,5.6.7.8 or 5.6.7.8,1.2.3.4 depending on the proxy's header order.

See also: https://github.security.telekom.com/2020/05/smuggling-http-headers-through-reverse-proxies.html

Because of this risk, many servers ignore any header whose name contains underscores.

• Apache 2.4+: https://httpd.apache.org/docs/2.4/new_features_2_4.html#module
  • but only for internal environment variables - used by mod_cgi, mod_wsgi, etc.
  • mod_proxy_http forwards all original headers, preserving spelling.
• Nginx: https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/#missing-disappearing-http-headers
  • can be overridden with underscores_in_headers on;.
• Django (CVE-2015-0219): https://www.djangoproject.com/weblog/2015/jan/13/security/
• Waitress: https://github.com/Pylons/waitress/commit/6d4dab6bed88917b973066a6d5222917661802b7
• Bjoern: https://github.com/jonashaag/bjoern/commit/9fd4d605cb7859632af7cc0f93b9a512510be66b

What should Gunicorn do about this?

• "Always ignore headers with underscores." - Probably a bad idea, I'm sure many Gunicorn users rely on the current behavior.
• "Add an option like Nginx's underscores_in_headers, default to ignore them." - My personal favorite option, but it is probably a major compatibility break. And it will definitely cause some confusion - every server that made this change has a ton of online questions asked about it.
• "Add an option like Nginx's underscores_in_headers, default to allow them." - Sad, but at least some users can opt-in.