acton icon indicating copy to clipboard operation
acton copied to clipboard

How to nicely structure async programs?

Open plajjan opened this issue 2 years ago • 2 comments

Got to discuss this. Current programs do not look very nice. Hard to read code etc. How do we structure async programs in a nice way?

It feels like server style apps are quite a good fit as-is. The problem is with a program that "wants" something. A server just responds to a query but most other tasks are more like a sequence of things to do.

  • get the bla
  • do the foo
  • compute the blah
  • send blah to baba

With the async nature of Acton, this is really tedious to write. Each call that does external I/O takes a callback function and yeah, it just gets very hard to read. I guess it's FSM style but it's hard to setup proper FSM and it's hard to reason about. Like, have I covered all cases in all states? Maybe we should have compiler support for FSM. It's really like pattern matching, which is something we do want good support for, but I think the syntax needs to look different.

Anyway, for a program that "wants something", a FSM does NOT seem like a very good way to express the sequence of steps. Like, anything could of course go wrong at any time, so it is important to handle failures etc, but if one considers it an FSM, I think there is a path of normal steps that are most common and this is what most imperative languages capture so well. We don't. Is there a silver bullet?

  • FSM?
  • JS .then() ?

plajjan avatar Aug 06 '23 12:08 plajjan

Here's one suggestion for how to solve this.

We would allow forward references from the calling location to the definition of the called function. The only thing that's weird is that it sort of looks like we can access "google_conn" from within on_google_connect because structurly google_conn is defined above and on a higher indent level than on_google_connect, but because on_google_connect is passed as the callback in a scope where google_conn is not yet defined, it is not reachable from within on_google_connect.

actor main(env):
    google_conn = http.HTTPClient(cap, "www.google.com", 80, on_connect=on_google_connect, timeout=)
    ai_conn = None

    def on_google_connect(c, err):
        # We cannot access google_conn here, as it isn't initialized yet. From a
        # purely text scope perspective it looks as it is defined, since it's on
        # a line above us, but since we are passed as a callback to HTTPClient,
        # which is the thing assigned to google_con, we are actually called
        # before google_conn is initialized.
        if c is not None:
            c.request("GET", "/q=asdf", on_search_response)
            
    def on_search_response(c, response):
        if response is not None:
            print "Status:", response.status
            print "Reason:", response.reason
            print "Headers:", response.headers
            hits = parse_search_results(response.body)
            ai_conn = http.HTTPClient(cap, "api.openai.com", 80, on_connect=on_ai_connect)

    def on_ai_connect(c, err):
        if c is not None:
            c.request("POST", "/v1/engines/davinci/completions", on_ai_response)

    def on_ai_response(c, response):
        if response is not None:
            print "Status:", response.status
            print "Reason:", response.reason
            print "Headers:", response.headers
            print "Body:", response.body
            # DO MORE stuff....

Is this restriction easy enough to understand? Is the program easy to write, read and understand?

The intent of the program aligns well with the source code:

  • connect to google
  • once connected, send a query
  • once we've gotten the query response, connect to openai
  • once connected to openai, send another query

The functions are exactly in that order, which is the key point behind this proposal.

plajjan avatar Aug 15 '23 12:08 plajjan

I like it. This is the exact thing I was suggesting in the call.

ciaran2 avatar Aug 15 '23 17:08 ciaran2