software-layer
software-layer copied to clipboard
distribute building of software layer across nodes in a cluster
We're currently always using a single node to build the software layer, it would be nice to have support for leveraging a pool of available nodes, which should significantly speed up the process.
TODOs:
- [ ] fully autonomous installation procedure using EESSI build container (incl. running in Gentoo Prefix environment)
- [ ] set up overlay directories in a shared filesystem (?)
- or use local filesystem and create tarballs per installation, which are expanded as-needed through an EasyBuild hook
- [ ] modify existing of add separate build script
- [ ] add support to EasyBuild to provide a template for a job script to allow building in a container image
Have you looked at using distcc? I never tried it on prefix, I wonder if that would work at least for the compatibility layer.