easybuild-framework icon indicating copy to clipboard operation
easybuild-framework copied to clipboard

add support for installing in new namespace with bwrap

Open smoors opened this issue 3 years ago • 5 comments
trafficstars

when reinstalling software with EB on an active cluster, the software is unavailable or in a broken state during the reinstallation process, which might take a long time

here we propose a 2-step install procedure to keep the time that the software is unavailable as short as possible:

  1. install the software in a new namespace in a different location with bwrap (bubblewrap)
  2. copy the reinstalled software to the original location, as fast and as "atomically" as possible

here is an example how bwrap can be used:

bwrap --bind / / --bind /path/to/bwrap/software/name /path/to/software/name --bind /path/to/bwrap/module/name /path/to/module/name --dev /dev --bind /dev/log /dev/log

the separate binds for software and modules are necessary in case of a non-default installation directory tree. for example, in our site it is organized as follows:

--software
    --soft1_name
        --version1
--modules
    --toolchain_generation1
        --all
            --soft1_name
                --version1

here is a way to copy the installation directory that is not fully atomic, but close. this assumes that both the bwrap and the original directory are in the same file system:

rsync -av --delete-after --link-dest=/path/to/bwrap/dir /path/to/bwrap/dir /path/to/original/dir

with --link-dest=, files are not copied but hardlinks are created to the bwrap dir

smoors avatar Oct 25 '22 07:10 smoors