vibora

vibora copied to clipboard

Good work!

I'm wondering what it'd take for you to be convinced in adopting ASGI as the interface between the server/application?

At the moment the Python asyncio landscape is in a place where framework authors are continually re-implementing low-level server handling and having the rest of the full-stack framework be tightly coupled to the server implementation.

We'd be in a far better place as a community if eg. Vibora's work on super-fast Cython request handling could be used to run any ASGI-compatible framework.

So far API Star and Quart have both adopted ASGI as the interface layer. Flask and Django are both considering doing so.

Uvicorn is an ASGI server that can serve either API Star, Quart, or Django Channels, outperforms Sanic, and currently tops the Techempower Fortunes benchmarks for Python frameworks. No doubt at all a Vibora-powered ASGI server would improve on that further.

Providing a unified interface for server implementations to run applications against with would be hugely beneficial, de-coupling server implementations from framework implementations, and would also allow us as a community to write middleware that runs across a range of different frameworks.

My 2 cents: while I agree on the fact that having a common server interface would be beneficial to all I think ASGI and uvicorn is not the response IMHO. The choice of ASGI to use dictionaries is largely a performance limiting decision on CPython.

Japronto chose densely packed but properly aligned C structs and it outperforms uvicorn by a factor of 4 https://www.techempower.com/benchmarks/#section=data-r16&hw=ph&test=plaintext&l=hr9ywv in plaintext benchmark.

The choice of ASGI to use dictionaries is largely a performance limiting decision on CPython.

If that’s a significant concern, then a server implementation could use dict-like objects for its ASGI messages, no?

My point wrt. uvicorn is less “use uvicorn” and more that introducing ASGI as the interface layer isn’t a significant factor. It’s existing uvloop+httptools implementation has sanic-like performance. A implemented-in-C-and-picohttpparser ASGI server would have implemented-in-C-and-picohttpparser-like performance. A Vibora inspired cython-all-the-way-through implementation would have cython-all-the-way-through-like performance.

(Aside: The plaintext benchmark is a poor case of comparison given that it enables HTTP pipelining, and skews massively towards pipelining implementations.)

I agree about plaintext and json benchmarks.

Maybe I am missing something but ASGI documentation shows dictionaries with strings, lookup involves hashing of strings that will be (depending on your luck) stored in different cache line. At some point I tested the performance of small dictionaries versus aligned C structs with possibly thinest cpy-ext shims over them to expose them to Python. The lack of hashing, data locality and predictable memory address patterns made a huge difference. The diff in number of CPU instructions per lookup on x86_64 was significant.

My main concern here is that ASGI design leaves too much freedom on memory management to CPython.

First of all, I'm a big fan of both of you! (you can be sure there is a lot of code inspired in yours hahaha) and I'm a bit ashamed that you got here in such an early stage of the framework :P

Now talking about ASGI: This idea was already under my radar. I have a few concerns about it because I want to optimize way further down as @squeaky-pl said and I'm not sure ASGI would let me... Vibora is currently lacking a lot of optimizations so I'll give ASGI a try before it's too late.

so I'll give ASGI a try before it's too late.

I think you've got a great opportunity here. I'm definitely willing to help put in time into helping out if needed.

I guess a first pass would be to implement cprotocol.pyx so that it calls into a plain old ASGI application, and look at the performance that gives you.

I have a few concerns about it because I want to optimize way further down as @squeaky-pl said and I'm not sure ASGI would let me

It's certainly possible that a strict interface between application/server might prevent some kinds of micro-optimizations. For instance I see you've got some handling of "lookup cached route" directly in the server codebase itself, that probably wouldn't be able to work quite that way anymore (tho you would perfectly well still be able to cache it application side.)

I think it's a bit hard to take a call on that without first getting a baseline for an ASGI server implemented against your protocol code.

Yes. I'll fork a version using ASGI to discuss with you. I have a few more doubts especially related to timeout handling because Vibora has route-based timeouts and route-based transfer limits (body size, headers size, etc)

Vibora has route-based timeouts and route-based transfer limits (body size, headers size, etc)

Right. Some of those things you might also be able to do application-side if you wanted to, tho I think the sensible approach would probably be to put any of those aside to start with and get something as simple as possible working initially.

Fair enough, many thanks for the support!

💯 It’s a massive amount of work you’ve done here. Very impressive.

The three of you should team up ;)

Is there still any interest in this?

@erm I'm getting used to ASGI and doing some POCs but can't promise anything :P

@andersea - I think it’s possible that you’re confusing “against asyncio” vs “against ASGI”. There’s plenty of valid criticism that the low level API for asyncio is complex, and also doesn’t encompass some of the nice aspects that trio and curio have later built on, but that’s largely completely separate from ASGI, where framework developers don’t have to care about any of the low level details anymore, and are just in async/await space.

If there is any communication that needs to happen, please let’s have actual discussion rather than “there are some negative comments”, which is completely uninformative.

Deleted some comments. As @tomchristie pointed out, they were not productive.

Well, it is arguably worth (briefly) considering the adoption of ASGI by frameworks such as Trio. From what I've understand, some Trio devs consider ASGI too asyncio-ish. But I don't think ASGI is limiting Trio in any way (there is a start here), and if it is, there is a good case for an ASGI 3.0. Regardless, ASGI is definitely a good idea to target :)

We’re a bit off topic here, but...

The ASGI spec is just an async/await interface. No strictly technical aspects there to prevent trio or curio implementations, eg. https://github.com/encode/uvicorn/pull/118

(Tho there are ecosystem reasons to prefer asyncio compatibility.)