Proposal: Fetch-like API for Revery

[bryphe]

One major gap we have at the moment with Revery is that there isn't a prescribed way to do HTTP/HTTPS requests. This is very important functionality to have!

There are two separate aspects here - the API surface we want to provide, and the underlying tech (both native/JS). I also believe it makes sense to factor this out to a separate library such that all the native-UI projects can benefit, as there is nothing revery specific here (ie, reason-fetch), but wanted to start the discussion here.

API Surface

For the API surface - following the theme of empowering ReactJS developers to use their skills for native - I believe the fetch API is a reasonable model. We might make some tweaks to use the language, ie:

fetch(~url="https://api.reddit.cokm", ~method="GET", ...., ~body="...");

A few open questions:

• How should the streaming behavior be modeled? To start, we could not implement this (to unblock simple requests / usages), and then revisit as the need arises.
• Promises are used to return the value - we could model this with Lwt promises w/o needing the Lwt engine - is there a better approach?

Implementation

For the JavaScript implementation, it would be straightforward to just use fetch, and coercing values as we need to fit back into JSOO's model.

For the native implementation, there are a lot more possibilities. Most of the approaches we looked at so far involve using OCaml libraries like http/af or similiar to implement this request functionality.

One challenge with these native implementations is they require an event loop - like lwt-based implementations require Lwt_unix and Lwt_main.run to provide this event loop. This is problematic for compiling to JSOO, because this model isn't well supported in that environment. It's also not clear exactly how to map this to Revery's 60hz update model - would it be possible to have an event loop that Revery 'pumps' every frame?

Another approach that we haven't spent much time investigating is - would it be possible to use a C/C++ native-based library, like libcurl, cpprestsdk, or libhttp, and wrap in OCaml stubs?

The nice thing about this - is instead of creating an event loop, we could run the networking code in a completely separate OS thread, and just callback into the runtime with the results. This would give us the same fetch API semantics as JS w/o needing to worry about implementing our own event loop. I'm curious if this would be feasible? If so, it's possible it could perform better, as off-loading that entirely to a separate C thread would keep the rendering thread free.

[rauanmayemir]

@bryphe Lwt_main.run would also be problematic because it blocks the thread. 🙁

[bryphe]

@bryphe Lwt_main.run would also be problematic because it blocks the thread. 🙁

@rauanmayemir - definitely. I was thinking / wondering if we could host a parallel Lwt event loop in another OCaml thread. If we used something like http/af or httpkit - we could delegate promises to run in that separate event loop, and callback. But I'm not sure it's ideal (and we still run into the problem of lack of JSOO compatibility).

At that point - it feels like we'd be better off just grabbing a c thread to handle the networking - so most of that work / waiting is handled concurrently with the OCaml runtime.

[rauanmayemir]

We did just that: briskml/brisk#39.

[rauanmayemir]

At that point - it feels like we'd be better off just grabbing a c thread to handle the networking - so most of that work / waiting is handled concurrently with the OCaml runtime.

I think you'd essentially be reimplementing Lwt in this case.

[hcarty]

I think some of these things exist in some form already:

• For integrating Lwt with another event loop, you can see an example for glib
• curl + OCaml: See ocurl. Plays well with Lwt and running in a separate thread. It's a pretty close binding to the underlying C API so a convenience layer on top would be very helpful.
• Lwt, in general, already works well with JSOO. There is no need for Lwt_main.run under JSOO as far as I understand.

[yunti]

There is definitely a great benefit in having this as a separate library and it would likely be a very important one in the reason / ocaml ecosystem. For now the default for targeting js seems to be bs-fetch (rather than a slightly higher level of abstraction such as bs-axios - which has some benefits eg https://medium.com/@sahilkkrazy/fetch-vs-axios-http-request-c9afa43f804e).
However I think in Reason we would be better using fetch with some thoughts about how to handle promises better such as using https://github.com/aantron/repromise .

The more difficult challenge is definitely the native side, but I think where possible we should stick with an API familiar to the js side (where appropriate) as that will be help ease more people into the ecosystem (from js).

[eschaefer]

I think the value in this would be immense for folks on the fence. Right now it's the only missing feature that stops us from adopting Revery over electron!

[lessp]

Hey @eschaefer, I've started the work on a Fetch-like API for ReasonML.

Revery could provide it within the framework for convenvience and customisations, but you can use it now, standalone as well, if you wish: https://github.com/lessp/fetch

"dependencies": {
  ...
  "fetch-native-lwt": "lessp/fetch:fetch-native-lwt.json"
}