HTTP 2 Support?

[jamadden]

gevent.pywsgi is an implementation of the WSGI spec for HTTP <= 1.1. However, HTTP 2 is now the "current" version of the HTTP spec and support is becoming more widespread. That leads to a number of questions, at least:

• Is the WSGI spec flexible enough to handle being served over HTTP 2? How about efficiently served?
• If so, is it necessary for gevent to include HTTP 2 support? Production uses of gevent to serve WSGI apps are often behind some kind of proxy for scaling and safety reasons, such as nginx or haproxy. Some best practices in that case call for them to do the HTTP heavy lifting and use the PROXY protocol to communicate with the backing server. gevent does not include an implementation of the PROXY protocol, but a widely used downstream project, gunicorn, does. This leaves the complexity to the front-end servers and keeps the gevent servers simple. (This also assumes that the front-ends continue to do the parsing and spoon feed the backend 1.1 compatible requests.)
  • Secondary question: should gevent implement the PROXY protocol? The PROXY protocol wraps HTTP requests and adding support isn't terribly complicated.
• If both of those are yes, how best to implement that support? HTTP 2 is a complex protocol and requires a substantial amount of code to fully implement. Can gevent take an (optional?) dependency on a third-party implementation of HTTP 2? (Currently I know of one, hyper-h2.)

[byaka]

I'm think it will be great to implement http2 support in gevent. First of all, every popular library (and gevent is very popular), that supports http2 - it's another step to future :) but maybe implement it with third-party library? for example https://github.com/lukasa/hyper

[parasyte]

I vote for leveraging Hyper. (As much as necessary. I assume gevent would only use its HTTP/2 parser, and perhaps a minimal subset of its runtime protocol support.) HTTP/2 in gevent is definitely going to be a welcomed addition, and deferring to another project for maintaining the complexity seems perfectly reasonable.

As far as adding PROXY protocol support, sure, why not! The spec is really quite basic: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt

That said, I'm unfamiliar with all of the peculiarities of WSGI...

[boyxuper]

Here's new library: hyper-h2 for only protocol layer. https://github.com/python-hyper/hyper-h2

[Omnifarious]

I would prefer a protocol engine that expected to speak that protocol to a generic asynchronous interface rather than speaking it directly to a socket. And it would be nice if that were true on both sides. I would like to be able to have a relatively generic asynchronous server to be able to talk to the HTTP/2 layer as well. But that sort of goes without saying and is how most people design things like this. The socket thing isn't.

[ghost]

I know you guys where looking for something that is just an interface to this from the last comment @Omnifarious left and it appears that the nghttp2 library might fit the bill for such a requirement? I didn't know if you guys looked at this or not. They have some pretty straightforward python bindings, and it should be something that is easy to bundle with the package.

Just wanted to throw it out there just in case. I'm currently experimenting with these bindings so I suppose I'll let you know what I find!

[byaka]

@ProtonScott interesting project, need to look it deeper..

[pkittenis]

Not sure WSGI - the web application interface layer - needs to support any particular version of HTTP. The web server handling requests should already be able to do that and pass on HTTP2 requests to WSGI via its socket interface, or however the webserver has chosen to implement WSGI. None of that requires HTTP2 support in gevent's implementation of WSGI, nor should gevent be handling client requests directly as it's not a web server.

Have in the past used HTTP2 via apache with WSGI and gevent. From the client's perspective that is HTTP2. How the web server communicates with the WSGI application is not really relevant to the client.

[arcivanov]

I would agree with that. H2 is basically a multiplexing of streams into a socket. If you demultiplex into individual WSGI requests, WSGI stays the same. This is irrespective of gevent or threading or MP.