easybuild-easyconfigs
easybuild-easyconfigs copied to clipboard
easybuilders
{compiler}[GCC/11.2.0] hipSYCL v0.9.2 w/ Python 3.9.6
Tested with the following Clang distribution:
##
# This file is an EasyBuild reciPY as per https://github.com/easybuilders/easybuild
#
# Copyright:: Copyright 2013-2015 Dmitri Gribenko, Ward Poelmans
# Authors:: Dmitri Gribenko <[email protected]>
# Authors:: Ward Poelmans <[email protected]>
# License:: GPLv2 or later, MIT, three-clause BSD.
# $Id$
##
name = 'Clang'
version = '13.0.1'
versionsuffix = '-CUDA-%(cudaver)s'
homepage = 'https://clang.llvm.org/'
description = """C, C++, Objective-C compiler, based on LLVM. Does not
include C++ standard library -- use libstdc++ from GCC."""
# Clang also depends on libstdc++ during runtime, but this dependency is
# already specified as the toolchain.
toolchain = {'name': 'GCCcore', 'version': '11.2.0'}
source_urls = ["https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s"]
sources = [
'llvm-%(version)s.src.tar.xz',
'clang-%(version)s.src.tar.xz',
'compiler-rt-%(version)s.src.tar.xz',
'polly-%(version)s.src.tar.xz',
'openmp-%(version)s.src.tar.xz',
# Also include the LLVM linker
'lld-%(version)s.src.tar.xz',
'libcxx-%(version)s.src.tar.xz',
'libcxxabi-%(version)s.src.tar.xz',
'clang-tools-extra-%(version)s.src.tar.xz',
'libunwind-%(version)s.src.tar.xz',
]
checksums = [
'ec6b80d82c384acad2dc192903a6cf2cdbaffb889b84bfb98da9d71e630fc834', # llvm-13.0.1.src.tar.xz
'787a9e2d99f5c8720aa1773e4be009461cd30d3bd40fdd24591e473467c917c9', # clang-13.0.1.src.tar.xz
'7b33955031f9a9c5d63077dedb0f99d77e4e7c996266952c1cec55626dca5dfc', # compiler-rt-13.0.1.src.tar.xz
'f4003e03da57b53bf206faadd0cf53f7b198c38498c605dec45743db23c10ad0', # polly-13.0.1.src.tar.xz
'6b79261371616c31fea18cd3ee1797c79ee38bcaf8417676d4fa366a24c96b4f', # openmp-13.0.1.src.tar.xz
'666af745e8bf7b680533b4d18b7a31dc7cab575b1e6e4d261922bbafd9644cfb', # lld-13.0.1.src.tar.xz
'2f446acc00bb7cfb4e866c2fa46d1b6dbf4e7d2ab62e3c3d84e56f7b9e28110f', # libcxx-13.0.1.src.tar.xz
'db5fa6093c786051e8b1c85527240924eceb6c95eeff0a2bbc57be8422b3cef1', # libcxxabi-13.0.1.src.tar.xz
'cc2bc8598848513fa2257a270083e986fd61048347eccf1d801926ea709392d0', # clang-tools-extra-13.0.1.src.tar.xz
'e206dbf1bbe058a113bffe189386ded99a160b2443ee1e2cd41ff810f78551ba', # libunwind-13.0.1.src.tar.xz
]
dependencies = [
# since Clang is a compiler, binutils is a runtime dependency too
('binutils', '2.37'),
('hwloc', '2.5.0'),
('libxml2', '2.9.10'),
('ncurses', '6.2'),
('CUDA', '11.4.1', '', True),
('GMP', '6.2.1'),
('Z3', '4.8.12'),
]
builddependencies = [
('CMake', '3.21.1'),
('Python', '3.9.6'),
('Perl', '5.34.0'),
('elfutils', '0.185'),
]
assertions = False
usepolly = True
build_lld = True
libcxx = True
enable_rtti = True
build_extra_clang_tools = True
skip_all_tests = True
moduleclass = 'compiler'
Note especially that we need to turn off assertions since hipSYCL has a tendency to touch the edges of LLVM which can erroneously signal errors.
@nordmoen: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/1927712187 Output from first failing test suite run:
ERROR: test_conflicts (test.easyconfigs.easyconfigs.EasyConfigTest)
Check whether any conflicts occur in software dependency graphs.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/easyconfigs/easyconfigs.py", line 325, in test_conflicts
self.assertFalse(check_conflicts(self.ordered_specs, modules_tool(), check_inter_ec_conflicts=False),
File "test/easyconfigs/easyconfigs.py", line 277, in ordered_specs
EasyConfigTest.resolve_all_dependencies()
File "test/easyconfigs/easyconfigs.py", line 215, in resolve_all_dependencies
cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 461, in resolve_dependencies
raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 324, in raise_error_missing_deps
raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: Clang/13.0.1-GCC-11.2.0-CUDA-11.4.1 (no easyconfig file or existing module found)'
======================================================================
ERROR: test_dep_graph (test.easyconfigs.easyconfigs.EasyConfigTest)
Unit test that builds a full dependency graph.
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/easyconfigs/easyconfigs.py", line 312, in test_dep_graph
dep_graph(fn, self.ordered_specs)
File "test/easyconfigs/easyconfigs.py", line 277, in ordered_specs
EasyConfigTest.resolve_all_dependencies()
File "test/easyconfigs/easyconfigs.py", line 215, in resolve_all_dependencies
cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 461, in resolve_dependencies
raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 324, in raise_error_missing_deps
raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: Clang/13.0.1-GCC-11.2.0-CUDA-11.4.1 (no easyconfig file or existing module found)'
======================================================================
ERROR: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/easyconfigs/easyconfigs.py", line 798, in test_dep_versions_per_toolchain_generation
for ec in self.ordered_specs:
File "test/easyconfigs/easyconfigs.py", line 277, in ordered_specs
EasyConfigTest.resolve_all_dependencies()
File "test/easyconfigs/easyconfigs.py", line 215, in resolve_all_dependencies
cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 461, in resolve_dependencies
raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 324, in raise_error_missing_deps
raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: Clang/13.0.1-GCC-11.2.0-CUDA-11.4.1 (no easyconfig file or existing module found)'
----------------------------------------------------------------------
Ran 14381 tests in 680.585s
FAILED (errors=3)
ERROR: Not all tests were successful
bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice you me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.
@casparvl: Request for testing this PR well received on login1
PR test command 'EB_PR=15074 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15074 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!
- exit code: 0
- output:
Submitted batch job 8225
Test results coming soon (I hope)...
- notification for comment with ID 1060688787 processed
Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).
Test report by @boegelbot FAILED Build succeeded for 0 out of 2 (2 easyconfigs in total) cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8 See https://gist.github.com/fd59c88d6cb2bd408af18992d3d965cc for a full test report.
I had to search a bit for the actual error in the gist of boegelbot's build, but I see:
== 2022-03-07 18:20:26,525 easyblock.py:4061 WARNING build failed (first 300 chars): Sanity check failed: no file found at 'lib/libomptarget-nvptx.a' in /project/boegelbot/Rocky8/haswell/software/Clang/13.0.1-GCCcore-11.2.0-CUDA-11.4.1
This comes from the EasyBlock sanity check (here). While I'm no expert, if it got put in the sanity check, I assume it's supposed to be there somewhere. @nordmoen any idea why that archive is not present at the expected location? I assume the build completed succesfully for you, do you have that archive in your installation directory?
My bad @casparvl I forgot to mention that I have been using easybuilders/easybuild-easyblocks#2684 as it seems that Clang 13 no longer produces the lib/libomptarget-nvptx.a library.
@nordmoen Ok, let's see if I can convince boegelbot to use that PR for the EasyBlock :) In the meantime, I'm also trying a local build. It failed with a different error, but I didn't upload a test report yet. Will do that now, and immediately take that EasyBlock into account also for my local build.
@boegelbot please test @generoso EB_ARGS=--include-easyblocks-from-pr=2684
@casparvl: Request for testing this PR well received on login1
PR test command 'EB_PR=15074 EB_ARGS="--include-easyblocks-from-pr=2684" /opt/software/slurm/bin/sbatch --job-name test_PR_15074 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!
- exit code: 0
- output:
Submitted batch job 8228
Test results coming soon (I hope)...
- notification for comment with ID 1062003197 processed
Message to humans: this is just bookkeeping information for me, it is of no use to you (unless you think I have a bug, which I don't).
Test report by @casparvl Using easyblocks from PR(s) https://github.com/easybuilders/easybuild-easyblocks/pull/2684 FAILED Build succeeded for 0 out of 2 (2 easyconfigs in total) software1.lisa.surfsara.nl - Linux debian 10.11, x86_64, Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz, Python 2.7.16 See https://gist.github.com/98dd0c5fa38174f5d451db1ffed78ce2 for a full test report.
Hm, the test failure on my system is because sys/cdefs.h is not in the /usr/include folder, but more deeply nested since it's multi-arch:
file /usr/include/x86_64-linux-gnu/sys/cdefs.h
/usr/include/x86_64-linux-gnu/sys/cdefs.h: C source, ASCII text
I'm not sure if something can/should be done about this at the EasyConfig level. I haven't run into this before with other EasyConfigs as far as I know. Then again, I also never installed CLANG before.
I'm also not sure which compilers is being used when this error is thrown: if it's CLANG itself (e.g. it's doing e.g. a bootstrap), then maybe the default include path should be changed (in which case it could be something that belongs in an EasyBlock). My GCC at least claims it looks in the correct include directory:
casparl@software1:~$ module load GCCcore/11.2.0
casparl@software1:~$ gcc -xc -E -v -
Using built-in specs.
COLLECT_GCC=gcc
OFFLOAD_TARGET_NAMES=nvptx-none
Target: x86_64-pc-linux-gnu
Configured with: ../configure --enable-languages=c,c++,fortran --without-cuda-driver --enable-offload-targets=nvptx-none --enable-lto --enable-checking=release --disable-multilib --enable-shared=yes --enable-static=yes --enable-threads=posix --enable-plugins --enable-gold=default --enable-ld --with-plugin-ld=ld.gold --prefix=/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0 --with-local-prefix=/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0 --enable-bootstrap --with-isl=/scratch/casparl/GCCcore/11.2.0/system-system/gcc-11.2.0/stage2_stuff --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.0 (GCC)
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'
/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0/libexec/gcc/x86_64-pc-linux-gnu/11.2.0/cc1 -E -quiet -v -imultiarch x86_64-linux-gnu - -mtune=generic -march=x86-64 -dumpbase -
ignoring nonexistent directory "/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0/include/x86_64-linux-gnu"
ignoring nonexistent directory "/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include-fixed"
ignoring nonexistent directory "/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include
/home/casparl/.local/easybuild/Debian10/2021/software/GCCcore/11.2.0/include
/usr/include/x86_64-linux-gnu
/usr/include
End of search list.
So it's hard to imagine gcc is throwing the error... thát should be able to find sys/cdefs.h in it's include path.
Test report by @boegelbot Using easyblocks from PR(s) https://github.com/easybuilders/easybuild-easyblocks/pull/2684 SUCCESS Build succeeded for 2 out of 2 (2 easyconfigs in total) cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8 See https://gist.github.com/4d5ab642c0613da0fa1cced45cdd51ed for a full test report.
@casparvl I have never seen that error before on the systems I have tested on, so I'm afraid I can't help much there.
At least boegelbot managed to build it properly.
@akesandgren will check on his system, which also has a multi-arch linux setup. See if we can confirm where this issue comes from. In the meantime, I'll try to build an older CLANG EasyConfig to see if that does work on our system. If it does, something must have changed somewhere...
There is a patch we should add to this too. The CMAKE_VERBOSE_MAKEFILE setting isn't propagated to the sub projects it builds. Testing it now.
The problem with not finding sys/cdefs.h smells like a bug in clang. A simple
#include <sys/cdefs.h>
fails with the offending clang (1st stage) when doing
.../Clang/13.0.1/GCCcore-11.2.0-CUDA-11.4.1/llvm.obj.1/bin/clang -E /tmp/t.c -o /tmp/t.o -target nvptx64
but works if I remove the "-target nvptx64". So something goes wrong there that worked before.
I've been trying to look at the Debian patches for Clang 13 but haven't found anything yet.
Actually, our Clang/11.0.1 also presents the same error in the above case and so does Clang/10.0.0 I.e. this is not something new, it just happens to hit the bootstrap step with 13.0.1 which it didn't do previously.
@akesandgren Any progress on this? It seemed like the issue has always been present and maybe should not block this inclusion?
Going to test a patch for this....
Patch failed to accomplish what I wanted... Back to the drawing board...
The problem is described here: https://groups.google.com/g/llvm-dev/c/71cdlaU-fG8
Adding gcc-multilib (from the OS) to my build container fixes the problem. I would argue that this is a bug in Clang, that it should not need the things gcc-multilib pulls in to correctly setup the associatedtoolchains so it will find the multiarch include paths when using -target nvptx I know where it goes wrong, but not where the original problem is.
Test report by @akesandgren Using easyblocks from PR(s) https://github.com/easybuilders/easybuild-easyblocks/pull/2684 SUCCESS Build succeeded for 2 out of 2 (2 easyconfigs in total) b-an02.hpc2n.umu.se - Linux Ubuntu 20.04, x86_64, Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, Python 3.8.10 See https://gist.github.com/cc9322d4ba05895cd0e233e336195ebb for a full test report.
Note: Build done with gcc-multilib installed from OS in my build container, resulting installation also requires gcc-multilib installed from the OS to actually work when compiling something with -target nvptx that includes some file that lives under /usr/include/x86_64-linux-gnu/
This must be a bug in llvm/clang that it doesn't properly detect that the host is multiarch. It should NOT require gcc-multilib for that.
The reason it works by adding gcc-multilib is that (for my test with sys/cdefs.h) is that libc6-dev-i386 adds a softlink for /usr/include/sys/cdefs.h -> ../x86_64-linux-gnu/sys/cdefs.h
So the "bug" in Clang remains that it doesn't properly detect that the host is multiarch.