dune Allow concurrent exec with watch mode

This change allows a limited version of dune exec to run at the same time as dune is running in watch mode. This allows users to run example programs without needing to stop their watch server. This works by sending messages to the RPC server to build the executable if necessary.

May 23 '25 06:05 gridbugs

Thanks for the PR, it'll be easier to review once #11833 is merged.

May 23 '25 11:05 Leonidas-from-XIV

I've added a bunch of https://github.com/ocaml/dune/pull/11833 into this PR. Once https://github.com/ocaml/dune/pull/11833 is merged this diff will get much smaller.

May 27 '25 06:05 gridbugs

@gridbugs Can you rebase, now that #11833 is merged?

Jun 06 '25 09:06 Leonidas-from-XIV

Will do. I was waiting for https://github.com/ocaml/dune/pull/11879 to be merged first as it changes a lot of code in exec.ml.

Jun 09 '25 02:06 gridbugs

@Leonidas-from-XIV all the prereqs for this are now merged. Can you please take a look.

Jun 11 '25 08:06 gridbugs

Very useful work, however I am concerned about the maintainability if the tests are disabled for being flaky. Is there maybe some way to make them more stable?

Flaky watch-mode tests is something that has plagued dune for as long as I've worked on it and I'm not sure what exactly we can do about it. @Alizter I recall you doing some work on this recently?

Jun 12 '25 03:06 gridbugs

@gridbugs I fixed it somewhat with https://github.com/ocaml/dune/pull/11827 and then it was decided a more general approach as in https://github.com/ocaml/dune/pull/11841 would be a better fit. The issue in the latter is that the CI is still a bit flaky for some reason.

Jun 12 '25 09:06 Alizter

This looks very nice, thanks a lot!! Would this MR solve https://github.com/ocaml/dune/issues/8114 too by any chance?

Oct 03 '25 16:10 yakobowski

@yakobowski the part of that relating to tests is being addressed in https://github.com/ocaml/dune/pull/12473

Oct 03 '25 17:10 shonfeder