Port to Eio and Cohttp

[aantron]

This is a messy draft port of Dream to Eio using cohttp-eio. On HTTP/1, it works better than the previous port attempts (#254, #194) and should be more maintainable, including because cohttp-eio offers a direct-style interface (the advantages of which this port still needs to propagate further into the Dream code), but a lot of features will be missing even after the port is complete -- HTTP/2 support, GraphQL, and others, because the corresponding libraries are not ported to Eio or not available with Cohttp. A lot of features, like Caqti integration, that will be available when this port is done, are temporarily disabled for now, and will be added back as the port progresses. WebSockets will require additional work.

The current status of the port is that I am working through various deadlocks and corner cases of using Eio "promises" together with fibers, which are caused by the existing code of Dream assuming a sort of promise-callback soup, where any callback can be called on the one big stack under Lwt.main and make progress, while with fibers and Eio blocking the wrong fiber to wait on a promise can block a stack that would have been used to make progress to resolve that promise. The semantics are therefore substantially different. This was a major problem with the approach in #194 and #254. It is substantially negated by thoroughly using direct style wherever possible, rather than promises or callbacks.

[smorimoto]

6.0 just shipped! https://github.com/mirage/ocaml-cohttp/releases/tag/v6.0.0

[yawaramin]

Trying to see if I can get this branch to run. There's been some bitrot, especially the Mirage folks have gotten rid of cstruct and moved to string/bytes. Hacking and slashing now...

EDIT: pushing my changes to https://github.com/yawaramin/dream/tree/cohttp-eio

The examples are running now. Will play around with it a bit more.

[kognise]

I think this is super important, is there any way I can help speed up the work to port dream to eio?

[clauverjat-h]

Hello @yawaramin

I tried to play with your fork, but I got the following error

Context: _private                        
File "src/http/dune", line 11, characters 2-10:
11 |   eio_main
       ^^^^^^^^
Error: Library "eio_main" not found.
-> required by library "dream.http" in _build/default/src/http
-> required by _build/default/META.dream
-> required by _build/install/default/lib/dream/META
-> required by _build/default/dream.install
-> required by alias install

To fix the issue I needed to replace eio_main by eio in src/http/dune

Note: I edited the message, I had other issues before but they were due to some misunderstanding on my part regarding dune package management, so nothing to do with your code.

[Willenbrink]

Just want to put out there that I have fixed some of the issues in yawaramin's fork which can be found here: https://github.com/Willenbrink/dream/tree/cohttp-eio.

All the tests pass and the examples we looked at work too. I've also rebased the changes on top of the additional tests I've written in #396 which also pass.

[yawaramin]

@Willenbrink great work! Checking out your fork.