easybuild-framework icon indicating copy to clipboard operation
easybuild-framework copied to clipboard

--sequential does not seem to perform a sequential build

Open Poshi opened this issue 2 years ago • 4 comments

I was trying to compile bash and it was failing. After some debugging, parallelism came as the possible cause of the issue. I tried to solve it by using --sequential, but it looks like it didn't worked, make was called with more than one job:

$ eb --prefix=/soft -r Bash-4.3-GCC-4.9.2.eb --try-software-version=5.1 --try-toolchain=GCCcore,12.1.0 --disable-map-toolchains --sequential
.
.
== FAILED: Installation ended unsuccessfully (build directory: /soft/build/Bash/5.1/GCCcore-12.1.0): build failed (first 300 chars): cmd " make  -j 256 " exited with exit code 2 and output:
.
.

Then I tried with --parallel=1 and it worked. But I was expecting the --sequential flag to do its job.

Poshi avatar Jun 16 '22 09:06 Poshi

I would expect --sequential to imply --parallel=1 but I can't find it used anywhere in framework!

ocaisa avatar Jun 16 '22 09:06 ocaisa

Found it: https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/tools/testing.py#L115

@boegel I think that the explanation for the use case in the --help is misleading. From looking at https://github.com/easybuilders/easybuild-framework/blob/develop/test/framework/toy_build.py#L2099 it seems that this option only means something in combination with --regtest

ocaisa avatar Jun 16 '22 09:06 ocaisa

Found it: https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/tools/testing.py#L115

@boegel I think that the explanation for the use case in the --help is misleading. From looking at https://github.com/easybuilders/easybuild-framework/blob/develop/test/framework/toy_build.py#L2099 it seems that this option only means something in combination with --regtest

I think it means that the easyconfigs that are being built in the regtest are built one easyconfig after another - rather than allowing multiple easyconfigs to be built at once.

I think having EB error if --sequential is used without --regtest is the way forward and this can helpfully point people at using --parallel=1.

branfosj avatar Jun 16 '22 09:06 branfosj

It could make sense from the point of view of "one build after another". But the documentation is quite misleading. It says: Specify this option if you want to prevent parallel build. And, in fact, I wanted to prevent parallel build of a specific easyconfig. Maybe it is just a matter of rewording the documentation.

BTW, there should also be a way to limit the parallelism of how many easyconfigs are built in parallel. I didn't checked if that option already exists.

Poshi avatar Jun 16 '22 10:06 Poshi