Question about async streams and general discussion about development direction

[termermc]

One thing that's been a consistent issue across almost all Nim webservers (besides Chronos's slow server) is that request bodies and responses can't be streamed. There's a module in the stdlib called asyncstreams which exposes a type called FutureStream. I've worked with it before, you can use it to stream a response body with AsyncHttpServer. The module is marked as unstable, but it's so simple that I doubt anything substantial will change in it.

What are your thoughts on exposing request bodies and response streams as FutureStream objects? Right now, request bodies are being buffered entirely before request callbacks are dispatched, and this poses a security risk toward the server, because all you have to do is send a huge request body and fill up the servers memory. In my opinion, addition of streaming requests and additional options for limits on request bodies would make HTTPX production-ready, or at least usable for a wider range of applications.

The second thing I would like to discuss is about the development direction of this project. I would like to see HTTPX become the de-facto HTTP server for Nim.

Current alternatives are:

• HTTPBeast (no Windows support)
• Mummy (not async, and therefore no safe way to stream requests)
• Chronos (slow, can't use asyncdispatch)
• Scorper (reasonably fast, but you can't use asyncdispatch)
• AsyncHttpServer (we all know it sucks)

With this in mind, I think HTTPX could become a great choice for users once it has some of its inherent limitations removed, such as requiring requests to be fully buffered instead of being streamed. However, being a fork of HTTPBeast, I would like to know what your thoughts are for diverging from it. Many usability improvements could be made if HTTPX isn't required to function exactly the same way as HTTPBeast, but I don't know your stance on this. If you are willing, I would love to contribute the changes I mentioned and work on improving the server.

If you are not interested in diverging a little bit from HTTPBeast, I'll have to fork the server to add those changes, but I would much rather contribute directly to HTTPX because further fragmentation is bad for an already small ecosystem.

Let me know what you think.