How to nicely structure async programs?
Got to discuss this. Current programs do not look very nice. Hard to read code etc. How do we structure async programs in a nice way?
It feels like server style apps are quite a good fit as-is. The problem is with a program that "wants" something. A server just responds to a query but most other tasks are more like a sequence of things to do.
- get the bla
- do the foo
- compute the blah
- send blah to baba
With the async nature of Acton, this is really tedious to write. Each call that does external I/O takes a callback function and yeah, it just gets very hard to read. I guess it's FSM style but it's hard to setup proper FSM and it's hard to reason about. Like, have I covered all cases in all states? Maybe we should have compiler support for FSM. It's really like pattern matching, which is something we do want good support for, but I think the syntax needs to look different.
Anyway, for a program that "wants something", a FSM does NOT seem like a very good way to express the sequence of steps. Like, anything could of course go wrong at any time, so it is important to handle failures etc, but if one considers it an FSM, I think there is a path of normal steps that are most common and this is what most imperative languages capture so well. We don't. Is there a silver bullet?
- FSM?
- JS .then() ?
Here's one suggestion for how to solve this.
We would allow forward references from the calling location to the definition of the called function. The only thing that's weird is that it sort of looks like we can access "google_conn" from within on_google_connect because structurly google_conn is defined above and on a higher indent level than on_google_connect, but because on_google_connect is passed as the callback in a scope where google_conn is not yet defined, it is not reachable from within on_google_connect.
actor main(env):
google_conn = http.HTTPClient(cap, "www.google.com", 80, on_connect=on_google_connect, timeout=)
ai_conn = None
def on_google_connect(c, err):
# We cannot access google_conn here, as it isn't initialized yet. From a
# purely text scope perspective it looks as it is defined, since it's on
# a line above us, but since we are passed as a callback to HTTPClient,
# which is the thing assigned to google_con, we are actually called
# before google_conn is initialized.
if c is not None:
c.request("GET", "/q=asdf", on_search_response)
def on_search_response(c, response):
if response is not None:
print "Status:", response.status
print "Reason:", response.reason
print "Headers:", response.headers
hits = parse_search_results(response.body)
ai_conn = http.HTTPClient(cap, "api.openai.com", 80, on_connect=on_ai_connect)
def on_ai_connect(c, err):
if c is not None:
c.request("POST", "/v1/engines/davinci/completions", on_ai_response)
def on_ai_response(c, response):
if response is not None:
print "Status:", response.status
print "Reason:", response.reason
print "Headers:", response.headers
print "Body:", response.body
# DO MORE stuff....
Is this restriction easy enough to understand? Is the program easy to write, read and understand?
The intent of the program aligns well with the source code:
- connect to google
- once connected, send a query
- once we've gotten the query response, connect to openai
- once connected to openai, send another query
The functions are exactly in that order, which is the key point behind this proposal.
I like it. This is the exact thing I was suggesting in the call.