microcluster icon indicating copy to clipboard operation
microcluster copied to clipboard

Add another hook for extended waitready operations

Open roosterfish opened this issue 4 months ago • 2 comments

In case a downstream of Microcluster (e.g. Micro*) does start some additional services besides the Microcluster daemon itself, there should be a way to be able to wait on those services to become ready. This effectively allows a user to wait until everything is ready so he can issue further commands/requests against the started services.

As part of its app struct Microcluster already has a Ready function which blocks until the daemon is ready. It returns right after the OnStart hook has finished. So someone could already start additional services as part of the OnStart hook and simply don't return until the services are ready. The hook already receives the daemon's shutdown context which won't be cancelled until the daemon is requested to shutdown through signals.

While this might already be a valid option, we should detach the "wait" operation from the general startup procedure to allow the daemon coming up quickly. Instead another hook PostStart(ctx) should be introduced which allows any downstreams to supply their own waitready logic. This new hooks will be called in its own routine right after the daemon closed its ready channel. It's then the duty of the downstream to exit this hook if everything is started or the retries are exceeded. But we should not limit the maximal amount of retries or set a fixed deadline on the provided context. Instead the hook receives the daemon's shutdown context so it can react properly.

roosterfish avatar Jul 30 '25 13:07 roosterfish

cc @mkalcok

roosterfish avatar Jul 30 '25 13:07 roosterfish

As discussed for MicroCeph it would be useful to be able to differentiate which service / condition exactly should be waited on, e.g.

  • microcephd ready once the microcephd is up
  • mon quorum formed once the mons have quorum
  • storage ready once the cluster can accept write i/o

sabaini avatar Aug 04 '25 15:08 sabaini