oneDNN
oneDNN copied to clipboard
brg:sve_256 fails benchdnn accuracy tests
Summary
test_benchdnn_modeC_matmul_multidims_cpu
is failing due to brg:sve_256
on main/v3.6 319a77eb3da6809476dbf0b512ffacfa88238b76
The failing benchdnn calls are
> ./tests/benchdnn/benchdnn --matmul --stag=acbd --wtag=acbd --dtag=acbd --attr-fpmath=bf16, 1x16x38x64:105x1x64x38
[23104][DST][1:0:0:0] exp_f32: -71 exp: -71 got: -157 diff: 86 rdiff: 1.21127
[23105][DST][1:0:0:1] exp_f32: 15 exp: 15 got: 8 diff: 7 rdiff:0.466667
[23106][DST][1:0:0:2] exp_f32: 186 exp: 186 got: -129 diff: 315 rdiff: 1.69355
[23107][DST][1:0:0:3] exp_f32: -3 exp: -3 got: -100 diff: 97 rdiff: 32.3333
[23108][DST][1:0:0:4] exp_f32: -1 exp: -1 got: 28 diff: 29 rdiff: 29
[23109][DST][1:0:0:5] exp_f32: 100 exp: 100 got: 40 diff: 60 rdiff: 0.6
[23110][DST][1:0:0:6] exp_f32: 15 exp: 15 got: -77 diff: 92 rdiff: 6.13333
[23111][DST][1:0:0:7] exp_f32: 95 exp: 95 got: -44 diff: 139 rdiff: 1.46316
[23112][DST][1:0:0:8] exp_f32: -62 exp: -62 got: 106 diff: 168 rdiff: 2.70968
[23113][DST][1:0:0:9] exp_f32: 31 exp: 31 got: 117 diff: 86 rdiff: 2.77419
[COMPARE_STATS][DST]: trh=1e-06 err_max_diff: 815 err_max_rdiff: 435 all_max_diff: 815 all_max_rdiff: 435
0:FAILED (errors:2396156 total:2425920) __REPRO: --matmul --stag=acbd --wtag=acbd --dtag=acbd --attr-fpmath=bf16 1x16x38x64:105x1x64x38
[23104][DST][1:0:0:0] exp_f32: -15580 exp: -15580 got: -34229 diff: 18649 rdiff: 1.19698
[23105][DST][1:0:0:1] exp_f32: 7185 exp: 7185 got: -1041 diff: 8226 rdiff: 1.14489
[23106][DST][1:0:0:2] exp_f32: 41612 exp: 41612 got: -27123 diff: 68735 rdiff: 1.65181
[23107][DST][1:0:0:3] exp_f32: 2969 exp: 2969 got: -15306 diff: 18275 rdiff: 6.15527
[23108][DST][1:0:0:4] exp_f32: 3112 exp: 3112 got: 8814 diff: 5702 rdiff: 1.83226
[23109][DST][1:0:0:5] exp_f32: 16910 exp: 16910 got: 12855 diff: 4055 rdiff:0.239799
[23110][DST][1:0:0:6] exp_f32: -401 exp: -401 got: -12707 diff: 12306 rdiff: 30.6883
[23111][DST][1:0:0:7] exp_f32: 25251 exp: 25251 got: -10682 diff: 35933 rdiff: 1.42303
[23112][DST][1:0:0:8] exp_f32: -12964 exp: -12964 got: 23692 diff: 36656 rdiff: 2.82752
[23113][DST][1:0:0:9] exp_f32: 5708 exp: 5708 got: 24813 diff: 19105 rdiff: 3.34706
[COMPARE_STATS][DST]: trh=1e-06 err_max_diff: 176270 err_max_rdiff: 60963 all_max_diff: 176270 all_max_rdiff: 60963
1:FAILED (errors:2402790 total:2425920) __REPRO: --matmul --stag=acbd --wtag=acbd --dtag=acbd 1x16x38x64:105x1x64x38
tests:2 passed:0 skipped:0 mistrusted:0 unimplemented:0 invalid_arguments:0 failed:2 listed:0
Pinging @vineelabhinav @Shreyas-fuj from git blame
Environment
- CPU: Neoverse-V1 C7g.4xlarge
- OS version: Ubuntu 20.04
- Compiler version: gcc-10
- git hash: 319a77eb3da6809476dbf0b512ffacfa88238b76
- CMake version: 3.16.3
- CMake output log
+ cmake -DDNNL_CPU_RUNTIME=OMP -DCMAKE_BUILD_TYPE=Release -DDNNL_BUILD_FOR_CI=ON -DDNNL_WERROR=OFF -DDNNL_TEST_SET=NIGHTLY ..
-- The C compiler identification is GNU 10.5.0
-- The CXX compiler identification is GNU 10.5.0
-- Check for working C compiler: /usr/bin/gcc-10
-- Check for working C compiler: /usr/bin/gcc-10 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-10
-- Check for working CXX compiler: /usr/bin/g++-10 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DNNL_TARGET_ARCH: AARCH64
-- DNNL_LIBRARY_NAME: dnnl
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Could NOT find Doxyrest (missing: DOXYREST_EXECUTABLE)
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.18", minimum required is "2.7")
-- Found Sphinx: /usr/bin/sphinx-build (found version "sphinx-build 1.8.5")
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Enabled testing coverage: NIGHTLY
-- Enabled workload: TRAINING
-- Enabled primitives: ALL
-- Enabled primitive CPU ISA: ALL
-- Enabled primitive GPU ISA: ALL
-- Enabled GeMM kernels ISA: ALL
-- Primitive cache is enabled
-- Graph component is enabled
-- Graph compiler backend is disabled.
-- Configuring done
-- Generating done
-- Build files have been written to: oneDNN/build