Race condition in "cabal run"

[chreekat]

Context:

I am working on a project that has dozens of packages and ~10 executables. I have created a process-compose.yml that allows me to run the executables in parallel. Each executable is run via cabal run the-executable for simplicity.

Problem:

If the executables need to be rebuilt, or more precisely, a dependency within the project need to be rebuilt, I regularly see errors that look like race conditions caused by a lack of locking. One such error looked like:

ghc-pkg-9.6.6: cannot create: /path/to/project/dist-newstyle/build/x86_64-linux/ghc-9.6.6/THE-DEPENDENCY-0.2.0.0/noopt/package.conf.inplace already exists

Obviously this particular error is coming from ghc-pkg, so you could argue it's a ghc-pkg bug. But I assume cabal is responsible for the invocations of ghc-pkg in the first place. As the coordinator, cabal should probably recognize when another instance is already building a given package, and wait.

Cabal version: 3.12.1.0

[andreasabel]

As the coordinator, cabal should probably recognize when another instance is already building a given package, and wait.

(In variant #11329, a similar race occurs when creating the .cabal/store for a new GHC for the first time. This seems to be beyond tracking building a given package, it is more like building a first package for a GHC.)

I have created a process-compose.yml that allows me to run the executables in parallel. Each executable is run via cabal run the-executable for simplicity.

@chreekat There might be other problems if you invoke several instance of cabal in parallel, especially when all of them want to write to the local build tree. How would you know that they want the same configuration of the dependencies they all are trying to build? You could start them with different -f options, for instance...