How to deal with async code in state start

[jpmonettas]

Hi, I'm using mount in ClojureScript and I'm having a hard time figuring out how to deal with async code inside a state start. How does component B that depends on A knows A is ready when A start code is async?

I have something like:

(ns server.db)

(defstate db
  :start (let [conn (create-db-connection)]
           (create-tables conn (fn []
                                 ;; here I know tables are created
                                 ))
           {:conn conn})
  :stop ...)

(ns server.syncer)

(defstate syncer
  ;; how can I synchronize this so I'm sure
  ;; this component is started when db component has finished starting
  :start (insert-some-stuff-in-db (:conn @server.db))
  :stop ...)

Thanks!!

[matheusemm]

Hello, @jpmonettas. I hope you already figured out how to solve your problem. If not, I was wondering if wrapping the initialization of the db state in a future would work. Something like:

(defstate db
  :start (future (let [conn...])))

(defstate syncer
  :start (let [conn @db] (insert-some-stuff-in-db (:conn @server.db))))

The downside of this approach would be using @db instead of just db when referring to the database.

[jpmonettas]

Hi @matheusemm , the problem with that is it doesn't work in ClojureScript where I need it now.

I ended up forking the project and made bringing states up and down wait on a value if the component start/stop fn returns a core.async/chan.

I'll try to change the solution so async is optional and submit a PR, maybe it works for other people.

[tolitius]

in the original example it looks like the syncer is a periodic "thing", if that is the case you can just check whether the connection exists before syncing:

boot.user=> (require '[mount.core :as mount :refer [defstate]]
                     '[yang.scheduler :as s])

;; "pretending" to take 3 seconds to create a DB connection
boot.user=> (defstate db :start (do (Thread/sleep 3000) {:connection 42}))
#'boot.user/db

boot.user=> (defstate syncer :start (s/every 500 
                                       #(if-let [conn (:connection db)]
                                         (println "syncing from" conn)
                                         (println "waiting for db connection")))
                             :stop (s/stop syncer))
#'boot.user/syncer

boot.user=> (mount/start #'boot.user/syncer)
{:started ["#'boot.user/syncer"]}

waiting for db connection
waiting for db connection
waiting for db connection
waiting for db connection
..

boot.user=> (mount/start #'boot.user/db)
waiting for db connection
waiting for db connection
waiting for db connection
waiting for db connection
waiting for db connection
waiting for db connection

{:started ["#'boot.user/db"]}

syncing from 42
syncing from 42
syncing from 42
syncing from 42

boot.user=> (mount/stop)
{:stopped ["#'boot.user/syncer"]}

if the syncer is not a periodic "thing", you can stop it after the first successful sync.

I don't think this is anything specific to how mount starts or stops states, but rather it is to how states interact with each other.

P.S. using yang scheduler to mock a potential syncer implementation. would be different in CLJS of course.

[jpmonettas]

Again, the problem I'm having is with using mount in ClojureScript where almost everything is async, so all the components that do IO at start will need all other components that depend on it to wait until it has finished before being able to start and use it's state.

I ended up forking and rewriting https://github.com/district0x/mount/commit/58a5d8e3350bea48e5ba01c62fe402f348f1e4e5 so that start/stop can optionally return a core.async/promise-chan, and the code that brings up/down components can wait when a state start has returned a promise to be resolved before starting the next one.

[tolitius]

right, I gave a Clojure example that is trivial to replicate in ClojureScript. Clojure REPL is just better to work with for examples.

here is a ClojureScript example:

cljs.user=> (require-macros '[mount.core :refer [defstate]])
cljs.user=> (require '[mount.core :as mount])

cljs.user=> (defstate db :start (let [conn (atom {})]
                                  (js/setTimeout #(reset! conn {:connection 42})
                                                 5000)
                                  conn))

cljs.user=> (defstate syncer :start (js/setInterval #(if-let [conn (:connection @@db)]
                                                       (println "syncing from" conn)
                                                       (println "waiting for db connection"))
                                                    1000))

cljs.user=> (mount/start)
{:started ["#'cljs.user/db" "#'cljs.user/syncer"]}

waiting for db connection
waiting for db connection
waiting for db connection
waiting for db connection
syncing from 42
syncing from 42
...

[jpmonettas]

I understand the example, but I don't think that the solution scales.

All components that need to use the db state should take into account that connection may no be ready yet. If you have a big tree of states, some of them with an async start (async is going to infect upwards in the dependency chain) you will have to poll (ask if ready) in each case, and if you forget one you will eventually end up with race conditions that are hard to debug.

[tolitius]

If you have a big tree of states

that would be something I would try to avoid. I tend to keep states as something really low level: I/O, threads, etc. a big tree of states would be something harder to reason about, hence debug to begin with

async is an interesting beast of its own. it "promises" ) to make things simpler, but it never does.