quicklisp-controller icon indicating copy to clipboard operation
quicklisp-controller copied to clipboard

Make the build process easy to parallelize

Open quicklisp opened this issue 12 years ago • 0 comments

In d2a68450e7b6acd027412f2364c3be1ac1622a08 @orivej made it possible to map over the source objects in parallel. Unfortunately, the build process of ensure-what-wins-you-can assumes it can use fixed locations on the filesystem as temporary storage for various build steps, like calculating making a dist tarball, computing the tarball's digest, and possibly other things. These steps clobber each other when run in parallel.

I'd like to make a new backend for ensure-what-wins-you-can that works within a sandboxed subdirectory for each system it tries to build.

The build functionality can be put into a separate binary that takes the system index, system name, and sandbox directory as command-line options, then does what it needs to put cache files, FASLs, temporary files, logs, output metadata, etc into the sandbox.

Then a certain number of those separate binaries can be run at the same time to reduce the amount of time the build takes overall.

quicklisp avatar Oct 31 '13 13:10 quicklisp