highway
highway copied to clipboard
google
riscv64: Illegal instruction / ImageTestGroup/ImageTest.TestAligned/Emu128
It seems there is an issue on riscv64 hardware and (un)aligned access:
99% tests passed, 2 tests failed out of 175
Total Test time (real) = 3.94 sec
The following tests FAILED:
169 - ImageTestGroup/ImageTest.TestAligned/Emu128 # GetParam() = 536870912 (ILLEGAL)
170 - ImageTestGroup/ImageTest.TestUnaligned/Emu128 # GetParam() = 536870912 (ILLEGAL)
Errors while running CTest
FAILED: CMakeFiles/test.util
ref:
- https://buildd.debian.org/status/fetch.php?pkg=highway&arch=riscv64&ver=0.17.1%7Egit20220711.f0a396a-1&stamp=1657562547&raw=0
(gdb) r "--gtest_filter=ImageTestGroup/ImageTest.TestAligned/Emu128"
Starting program: /home/malaterre/highway-0.17.1~git20220711.f0a396a/obj-riscv64-linux-gnu/tests/image_test "--gtest_filter=ImageTestGroup/ImageTest.TestAligned/Emu128"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/riscv64-linux-gnu/libthread_db.so.1".
Running main() from ./googletest/src/gtest_main.cc
DEBUG: getauxval returned 112d
DEBUG: getauxval2 returned 0
DEBUG: getauxval returned 112d
DEBUG: getauxval2 returned 0
Note: Google Test filter = ImageTestGroup/ImageTest.TestAligned/Emu128
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from ImageTestGroup/ImageTest
[ RUN ] ImageTestGroup/ImageTest.TestAligned/Emu128
Program received signal SIGILL, Illegal instruction.
hwy::ImageBase::ImageBase (this=0x3fffffdb28, xsize=1, ysize=1, sizeof_t=1) at /home/malaterre/highway-0.17.1~git20220711.f0a396a/hwy/contrib/image/image.cc:88
88 bytes_per_row_ = BytesPerRow(xsize, sizeof_t);
(gdb) x/i $pc
=> 0x3ff7fd5b40 <_ZN3hwy9ImageBaseC2Emmm+66>: vsetvli a4,zero,e8,m1,ta,mu
(gdb) bt full
#0 hwy::ImageBase::ImageBase (this=0x3fffffdb28, xsize=1, ysize=1, sizeof_t=1) at /home/malaterre/highway-0.17.1~git20220711.f0a396a/hwy/contrib/image/image.cc:88
No locals.
#1 0x0000002aaaabba62 in hwy::Image<unsigned char>::Image (this=0x3fffffdb28, xsize=1, ysize=1) at /home/malaterre/highway-0.17.1~git20220711.f0a396a/hwy/contrib/image/image.h:165
No locals.
#2 hwy::N_EMU128::TestAlignedT::operator()<unsigned char> (this=<optimized out>) at /home/malaterre/highway-0.17.1~git20220711.f0a396a/hwy/contrib/image/image_test.cc:50
img = {<hwy::ImageBase> = {xsize_ = 1, ysize_ = 1, bytes_per_row_ = 0, bytes_ = std::unique_ptr<unsigned char []> = {get() = 0x0}}, <No data fields>}
x = <optimized out>
y = <optimized out>
xsize = 1
ysize = 1
d = <optimized out>
rng = {static word_size = 32, static state_size = 624, static shift_size = 397, static mask_bits = 31, static xor_mask = 2567483615, static tempering_u = 11, static tempering_d = 4294967295,
static tempering_s = 7, static tempering_b = 2636928640, static tempering_t = 15, static tempering_c = 4022730752, static tempering_l = 18, static initialization_multiplier = 1812433253,
static default_seed = 5489, _M_x = {129, 1875655654, 2129871141, 1003550519, 1681848247, 3445760211, 821445142, 824170677, 3044826225, 850330728, 2806015250, 4125978459, 1118504388, 401781702,
3344950060, 61982874, 229936850, 2288564971, 1169649407, 3702962249, 515657542, 2577513651, 4288752363, 652481951, 317169363, 3307574360, 1454717697, 3474147099, 357491860, 1350525569, 2675465374,
2890925483, 1111010509, 1047091357, 2823224083, 1775511256, 3849090241, 2543051439, 543294311, 3327030986, 1174761077, 2104272877, 3375336518, 3283502180, 341121999, 2820587480, 4290992944,
1984317006, 2239956059, 1953716302, 1777147485, 1414542975, 3594579434, 340692514, 1207715232, 993108412, 2789632868, 1091302775, 2387475912, 169622253, 2972128445, 4089341592, 1095003749,
2479821235, 1224782869, 2318516261, 3666259621, 421772737, 2817336937, 1527540604, 1037520279, 1905814490, 3117924527, 150378890, 3963317692, 2506523558, 2539697152, 1032460055, 2205641569,
2051692894, 2395085259, 3001007518, 1308773086, 554819918, 2931991066, 2855324621, 687550465, 1406237116, 1005989097, 3968297286, 3315955091, 2099343915, 460358894, 352964675, 2200438221,
2794394122, 626025864, 3854948105, 2691182932, 2900131153, 2740182307, 1230235498, 1289719965, 3168504307, 1896244605, 439969365, 3099319539, 569517184, 4187023084, 642384824, 1899556870,
2820370994, 1648344416, 4179992758, 801916123, 1919770074, 2707241691, 3056949522, 2670048198, 2440024267, 3109548229, 3103620604, 1046586288, 2532230379, 32727401, 4194672618, 1378535531,
2198326353, 2024867903, 3170070263, 924771371, 2826438778, 2401996764, 1665110299, 477178312, 1283032175, 986848750, 3765185903, 1022408742, 4002348425, 3403604478, 986353246, 615283108,
1218495043, 3610244762, 3675466734, 3767347731, 161753315, 2312805667, 2570954650, 690714510, 2919442845, 2413724499, 2588354702, 513403862, 751407625, 1424781097, 4074521445, 2428644828,
4030326581, 3725467374, 4271507746, 227492775, 1380651142, 187090151, 4227616456, 1011005885, 1273261368, 941725989, 3963173186, 3472777295, 2113448615, 2029529642, 3550914212, 2994741137,
3732349102, 1714911729, 2578746465, 3725680321, 1474180413, 1506549088, 1659339002, 1413035709, 2818651363, 1273841277, 750332837, 2330756563, 933537072, 3568032940, 3182475464, 2938307184,
4265408953, 1003032866, 7140651, 1381875129, 2039173723, 3145705542, 2487601049, 2962666477, 289989906...}, _M_p = 624}
dist = {_M_param = {_M_a = 0, _M_b = 16}}
#3 0x0000002aaaab9d34 in hwy::N_EMU128::ForUnsignedTypes<hwy::N_EMU128::TestAlignedT> (func=...) at /home/malaterre/highway-0.17.1~git20220711.f0a396a/hwy/tests/test_util-inl.h:531
No locals.
#4 hwy::N_EMU128::TestAligned () at /home/malaterre/highway-0.17.1~git20220711.f0a396a/hwy/contrib/image/image_test.cc:69
No locals.
#5 0x0000002aaaae3eda in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) ()
No symbol table info available.
#6 0x0000002aaaad9398 in testing::Test::Run() ()
No symbol table info available.
#7 0x0000002aaaad94c8 in testing::TestInfo::Run() ()
No symbol table info available.
#8 0x0000002aaaad9928 in testing::TestSuite::Run() ()
No symbol table info available.
#9 0x0000002aaaadceb4 in testing::internal::UnitTestImpl::RunAllTests() ()
No symbol table info available.
#10 0x0000002aaaae4328 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ()
No symbol table info available.
#11 0x0000002aaaad9554 in testing::UnitTest::Run() ()
No symbol table info available.
#12 0x0000002aaaab97d4 in main ()
No symbol table info available.
It is not clear how vsetvli intructions appears in this context...this is supposed to be a Vector extension instructions...
Thanks, the gdb output is very helpful. vsetvli is typically the first vector instruction that we would execute. I'd think this is again due to a CPU that lacks the V extension. It would be great to to get the runtime dispatch support in so we could avoid crashing like this. I can go for that unless you want to?
Current runtime dispatch for RVV is done following this:
- https://github.com/google/highway/issues/838#issuecomment-1206529805
It appears that cxxflags -march=rv64gcv1p0 -menable-experimental-extensions are applied project-wide instead of per-source file. It means that instruction such as vsetvli could be found anywhere within image.o object file, this prevent proper runtime dispatch for RVV.
Please restrict -march=rv64gcv1p0 -menable-experimental-extensions to the minimal set of source files.
I can reproduce the isse on read-hardware.
$ cat /proc/cpuinfo
processor : 0
hart : 2
isa : rv64imafdc
mmu : sv39
uarch : sifive,bullet0
processor : 1
hart : 1
isa : rv64imafdc
mmu : sv39
uarch : sifive,bullet0
processor : 2
hart : 3
isa : rv64imafdc
mmu : sv39
uarch : sifive,bullet0
processor : 3
hart : 4
isa : rv64imafdc
mmu : sv39
uarch : sifive,bullet0
This breaks compilation on riscv64:
- https://buildd.debian.org/status/fetch.php?pkg=highway&arch=riscv64&ver=1.0.1%7Egit20220802.5810c58-4&stamp=1659695164&raw=0
One challenge we have is that we have to use static dispatch at the moment because we compile with clang, which doesn't yet support the target attributes for Arm and RISC-V.
-menable-experimental-extensions is already behind a clang-only check, we could also move the rv64gcv1p0 into that. Would that work for you?
@jan-wassenberg I just found out I cannot simply pass compile options to header file:
- https://stackoverflow.com/questions/48162570/cmake-how-to-set-compile-flags-for-header-files
instead I need to find all source file (cpp) which would need the special flag:
- https://stackoverflow.com/questions/13638408/override-compile-flags-for-single-files
I do not want to delay 1.0.1, so please go ahead and tag. I'll try to work on this ASAP.
OK :) But it's only a tiny change. We could add an opt-out CMake option for you that skips the rv64gcv1p0? (This would be used until runtime dispatch works, for anyone who wants to run on hardware that does not support V.)
OK :) But it's only a tiny change. We could add an opt-out CMake option for you that skips the rv64gcv1p0? (This would be used until runtime dispatch works, for anyone who wants to run on hardware that does not support V.)
True, but I do not know the full list of TUs to include in:
Please restrict -march=rv64gcv1p0 -menable-experimental-extensions to the minimal set of source files.
The easy way to skip rv64gcv1p0 is simply to build with gcc. So this is not an 'issue' for me on Debian.
I suspect we either want V in all files, or none - there's not much point to Highway nor its tests without some kind of vectors.
Here is what's currently in CMakeLists:
if(HWY_RISCV)
list(APPEND HWY_FLAGS -march=rv64gcv1p0)
if(${CMAKE_CXX_COMPILER_ID} MATCHES "Clang")
list(APPEND HWY_FLAGS -menable-experimental-extensions)
endif()
endif()
I don't yet understand why this is skipped when building with GCC? But if it is, that's good - it would have been my proposal to move rv64gcv1p0 into the clang section, which should be fine in terms of backwards compat because GCC anyway doesn't work at the moment with this flag and I don't know any other compiler that does.
I suspect we either want V in all files, or none - there's not much point to Highway nor its tests without some kind of vectors.
The whole point of this thread is that the above fails on unmatched riscv64 board (no v-extension):
- https://buildd.debian.org/status/fetch.php?pkg=highway&arch=riscv64&ver=1.0.1%7Egit20220802.5810c58-4&stamp=1659695164&raw=0
So how do you plan to handle runtime dispatch then ?
Yes, I understand :) For this board, and when runtime dispatch is fully supported, I think we will want to compile all files without the v compiler flag (which tells the compiler to assume its presence). Currently, we only want rv64gcv1p0 1) when compiling with clang 2) when the user knows the HW/emulator support V. Hence the proposal to make it conditional on clang - this would continue to work for our use case, whereas for your GCC build it will disable V.
(To clarify: once runtime dispatch is supported, we no longer require compiler flags and instead rely on #pragma to achieve the same effect.)
Hence the proposal to make it conditional on clang - this would continue to work for our use case, whereas for your GCC build it will disable V.
In that case no need to change a bit of the code. I'll revert my Debian package on riscv64 to use GCC instead of clang-14. Thanks for the clarification.
Here is what's currently in CMakeLists:
if(HWY_RISCV) list(APPEND HWY_FLAGS -march=rv64gcv1p0) if(${CMAKE_CXX_COMPILER_ID} MATCHES "Clang") list(APPEND HWY_FLAGS -menable-experimental-extensions) endif() endif()
My bad. I failed to read the above properly. It turns out you were right (as usual), this flag conflicts with our riscv64/baseline expectation at Debian:
- https://buildd.debian.org/status/fetch.php?pkg=highway&arch=riscv64&ver=1.0.1-1&stamp=1661418918&raw=0
Do you want me to do a PR with:
% git diff
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 81361b7..5572777 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -225,8 +225,8 @@ else()
endif() # HWY_CMAKE_ARM7
if(HWY_RISCV)
- list(APPEND HWY_FLAGS -march=rv64gcv1p0)
if(${CMAKE_CXX_COMPILER_ID} MATCHES "Clang")
+ list(APPEND HWY_FLAGS -march=rv64gcv1p0)
list(APPEND HWY_FLAGS -menable-experimental-extensions)
endif()
endif()
:) Sorry to be unclear. No worries, I've implemented this with an extra comment to explain what's happening 👍