Re-evaluate "http.response.zerocopysend"

[Archmonger]

trafficstars

Right now, none of the existing ASGI webservers support zerocopysend. This seems to be due to the additional code complexity needed to support this extension.

This lack of support has prevented me from adding zerocopysend to whitenoise, which would have been a big win for the Python web development community.

I'm opening this issue as a place to discuss what, if anything, can be done to encourage adoption for zerocopysend.

Here's a few questions to get this discussion started:

1. Is a zerocopysend2 interface needed to encourage adoption?
2. Does asgiref need some helper utilities to simplify adoption of the current extension?
3. It seems that adding zerocopysend would require facing the buffer. Is it worthwhile (or feasible) to engineer a way of avoiding this?

[Archmonger]

cc: @pgjones @Kludex @tomchristie

[Kludex]

The ones who know more about it are @synodriver and @abersheeran.

[abersheeran]

The reason for facing buffers is because of the design of asyncio. It seems possible to solve this problem if we modify the zerocopysend standard so that it can only be sent at the end of the message.

[Archmonger]

@abersheeran

In terms of what this would like in practice, do you have a proposed interface?

Trying to work backwards and see how/if to push through this issue.

Lack of zero copy send is pretty painful for Whitenoise and ReactPy.

[synodriver]

I have a fork of hypercorn with this extension, but it will only work under http1.1, since h2 doesn't provide this feature. As for facing the buffer, I personally think there's no problem because mine has been running for a long time.

[Archmonger]

@synodriver

Would an interface change allow h2 support without waiting on library support?

[synodriver]

@synodriver

Would an interface change allow h2 support without waiting on library support?

I don't think it's possible. hypercorn relys on those sans-io projects to parse raw data. You can send data directly, but that will break h2 library's internal state machine.

[Archmonger]

My immediate family impression it's debatable whether http2 and http3 file transfers need to be funneled through external libraries.

It seems possible to simplify things by adding a few asgiref helper functions for that.