Dealing with failure on mount/start

[lfn3]

At the moment there isn't a lot of guidance on how to deal with exceptions inside defstate start/stop methods.

My case is mostly related to bad config - if a user provides bad arguments to a program (specifically a directory that is supposed to contain certain files, and if those files are missing parts of the program will not be able to function.) how can we deal with some state being bad?

@tolitius suggested (in slack) wrapping the start method in a (try ... (catch ... )) which works if a default value can be supplied, however in many cases that may not be possible (database connection failure, for example)

In the database connection case currently I'm considering still using the try-catch and modifying application routes to prevent some actions, or changing having the defstate hold false or nil value to indicate failure, and having reliant parts of the application check the value before doing work. This pushes that concern out across large parts of the system, especially since this is largely affects the views - we want to disable buttons and provide error hints, for instance.

[tolitius]

can think of several ways of doing this with a caveat

Without Changes to Mount

Fail with default value

(defn start-foo []
  (try ...
    (catch Throwable t
      :dafault-foo-value)))

(defstate foo :start (start-foo))

Start "ok to fail" last

(mount/start-without #'app/foo #'app/bar)
(mount/start)

With Changes to Mount

A set of states to ignore

something like:

(-> (skip-failed #{#'app/foo #'app/bar})
    mount/start)

or

(mount/start {:skip-failed #{#'app/foo #'app/bar}})

Ignore failures per state

something like:

(defstate foo :start (start-foo)
              :skip-failed? true)

(not loving this one, since it "adds" to a clean :start / :stop API)

^^^ thinking out loud.

The Big Problem

Transitive failures.

Say we have 3 states A, B and C. On (mount/start), B failed to start. In case C depends on B, logically, it should not even attempt to start.

But the problem is there is no sure way to infer that dependency, since dependencies are really namespace based + all bets are off in ClojureScript.

So this needs more thinking.

What to do

We can probably start with:

(-> (skip-failed #{#'app/foo #'app/bar})
    mount/start)

since it is both: not current API / DSL changing and skip-failed is a single responsibility function: simple to add / remove, etc.

The transitive failure will not work auto magically, but can definitely be handled by just including a dependent state in skip-failed.

What do you think?

[lfn3]

I'm not sure yet, to be honest. At the moment I'm dealing with it by adding some pretty significant stuff inside the start catch, and that does work, although I'm not entirely happy with it, but I'm not sure that (skip-failed ...) is a massive improvement. I'm gonna hammock on it at bit and see if we can't come up with something better.

[CRHough]

I just ran into this but haven't dug into the source yet. Please forgive my ignorance if the following isn't feasible.

Could something like swap-states be done? Perhaps swap-on-fail that takes alternates to start on failures. Existing behavior doesn't change if there is no on-fail alternate.

I like the symmetry of handling alternates the same way whether they are from choice, testing, or failure.

How does that feel?

@lfn3 I solved the config issue by providing (hopefully) sane defaults in place of missing entries. The db connection failure I "solved" (unsatisfactorily) by breaking the starting into several groups (e.g., starting the states db depends on, checking that they started and starting db if so, checking that it started then starting what depends on db, etc.) much like tolitius' :default-foo-value. Neither is ideal but at least checking the state only happens in the one startup place rather than in each dependent state.

[tolitius]

I am still contemplating whether mount is the one that has to deal with state start/stop functions failures.

On one hand, it could relate to mount since it starts and stops states and may provide on-fail hooks. On the other hand mount should not interfere on failure, since failures have very different meanings in different contexts.

In case an app needs to know whether a state was :started / :stopped at runtime, there is a way to do it. If this is something that you find helpful, I could expose it in cljs as well: names and states, not the dependencies.

[CRHough]

mount deals with start and stop and if that was all I'd say error handling is not its responsibility. But it also aims to deal with dependencies. Maybe I'm using it wrong but having to write extra code to make sure the nodes are valid in addition to being properly ordered seems to me like the sort of thing the library should do.

On the other hand I can see an argument for dependency graph creation being separated from graph validation with life cycle separate from both others. If that helps with maintenance and improvement of the library code I'm all for it. I'm not sure the library is big enough to need it yet and it can always be refactored if and when the need arises.

---

After giving things some more thought if an alternate state can't be swapped in when an error occurs the system should at least not continue in the case of error.

If the point is still to "[Solve] 'application state' in Clojure" then can we really leave the application in a nebulous part up, part down state where it is likely not useful?

If dependencies are ignored this isn't really about application state. Instead it becomes a very clever way to call several functions from within another.

I really like this library and it works well. . . in the simplest case.

[tolitius]

@houghcr, I'd like to understand better what you are getting at.

when an error occurs the system should at least not continue in the case of error

In case a state fails to start: the whole app would not start by default, since the exception will be thrown.

This would probably be solved better if the problem is well defined. Could you share a concrete example where you don't think mount can do better reacting to state start failures?

[CRHough]

@tolitius I was doing something wrong. Everything works exactly the way it should. while building a minimal example, which I should have done before posting, I found I was eating the exception. I should have checked more thoroughly the mistake wasn't mine.

Please forgive the noise and feel free to delete anything not directly related to swap-on-fail.

[tolitius]

@houghcr great, thanks for the feedback. if you are still interested in swap-on-fail, could you share an example where it would be useful?

[CRHough]

@tolitius Absolutely.

The project I've just converted to mount connects to a remote server for a download. If the server isn't available the state fails (and, now that I'm finished being dumb, publicly, so does the system). If there is an alternate download location (in another state that normally isn't started) it would be wildly convenient if Mount let me fail forward so to speak.

Another place is with resource file reading. If the expected file doesn't exist then default data can be used and the expected file created in an alternate state.

Not to mention @lfn3's database connection.

[tolitius]

@CRHough recently I added mount-up :wrap-in custom functions that wrap start/stop calls.

This allows for a better control: exception handling, conditional fallbacks, etc.

For example here try-catch takes an on-error function that takes an exception and a state name and is called in case an exception is thrown on start/stop invocation, hence something like (pseudocode):

 (defn fallback [ex state]
   (case state
     "#'app.remote/server" (mount/start #'app.local/server)))

(mu/on-up :guard (mu/try-catch fallback :wrap-in)

is possible.

[pekeler]

I want my app to stop if a component throws an exception in :start. So if I have states A, B, C, D, E, and if for example C throws an exception in :start, I'd like B to be stopped, then A to be stopped, and then exit the process. What would be the simplest way to accomplish this? Would this be legal?

:start
 (try ...
  (catch Throwable t
   (mount/stop)))