oneDNN icon indicating copy to clipboard operation
oneDNN copied to clipboard

brg:sve_256 fails benchdnn accuracy tests

Open jondea opened this issue 7 months ago • 0 comments

Summary

test_benchdnn_modeC_matmul_multidims_cpu is failing due to brg:sve_256 on main/v3.6 319a77eb3da6809476dbf0b512ffacfa88238b76

The failing benchdnn calls are

> ./tests/benchdnn/benchdnn --matmul --stag=acbd --wtag=acbd --dtag=acbd --attr-fpmath=bf16, 1x16x38x64:105x1x64x38
[23104][DST][1:0:0:0] exp_f32:         -71 exp:         -71 got:        -157 diff:      86 rdiff: 1.21127
[23105][DST][1:0:0:1] exp_f32:          15 exp:          15 got:           8 diff:       7 rdiff:0.466667
[23106][DST][1:0:0:2] exp_f32:         186 exp:         186 got:        -129 diff:     315 rdiff: 1.69355
[23107][DST][1:0:0:3] exp_f32:          -3 exp:          -3 got:        -100 diff:      97 rdiff: 32.3333
[23108][DST][1:0:0:4] exp_f32:          -1 exp:          -1 got:          28 diff:      29 rdiff:      29
[23109][DST][1:0:0:5] exp_f32:         100 exp:         100 got:          40 diff:      60 rdiff:     0.6
[23110][DST][1:0:0:6] exp_f32:          15 exp:          15 got:         -77 diff:      92 rdiff: 6.13333
[23111][DST][1:0:0:7] exp_f32:          95 exp:          95 got:         -44 diff:     139 rdiff: 1.46316
[23112][DST][1:0:0:8] exp_f32:         -62 exp:         -62 got:         106 diff:     168 rdiff: 2.70968
[23113][DST][1:0:0:9] exp_f32:          31 exp:          31 got:         117 diff:      86 rdiff: 2.77419
[COMPARE_STATS][DST]: trh=1e-06 err_max_diff:     815 err_max_rdiff:     435 all_max_diff:     815 all_max_rdiff:     435
0:FAILED (errors:2396156 total:2425920) __REPRO: --matmul --stag=acbd --wtag=acbd --dtag=acbd --attr-fpmath=bf16 1x16x38x64:105x1x64x38
[23104][DST][1:0:0:0] exp_f32:      -15580 exp:      -15580 got:      -34229 diff:   18649 rdiff: 1.19698
[23105][DST][1:0:0:1] exp_f32:        7185 exp:        7185 got:       -1041 diff:    8226 rdiff: 1.14489
[23106][DST][1:0:0:2] exp_f32:       41612 exp:       41612 got:      -27123 diff:   68735 rdiff: 1.65181
[23107][DST][1:0:0:3] exp_f32:        2969 exp:        2969 got:      -15306 diff:   18275 rdiff: 6.15527
[23108][DST][1:0:0:4] exp_f32:        3112 exp:        3112 got:        8814 diff:    5702 rdiff: 1.83226
[23109][DST][1:0:0:5] exp_f32:       16910 exp:       16910 got:       12855 diff:    4055 rdiff:0.239799
[23110][DST][1:0:0:6] exp_f32:        -401 exp:        -401 got:      -12707 diff:   12306 rdiff: 30.6883
[23111][DST][1:0:0:7] exp_f32:       25251 exp:       25251 got:      -10682 diff:   35933 rdiff: 1.42303
[23112][DST][1:0:0:8] exp_f32:      -12964 exp:      -12964 got:       23692 diff:   36656 rdiff: 2.82752
[23113][DST][1:0:0:9] exp_f32:        5708 exp:        5708 got:       24813 diff:   19105 rdiff: 3.34706
[COMPARE_STATS][DST]: trh=1e-06 err_max_diff:  176270 err_max_rdiff:   60963 all_max_diff:  176270 all_max_rdiff:   60963
1:FAILED (errors:2402790 total:2425920) __REPRO: --matmul --stag=acbd --wtag=acbd --dtag=acbd 1x16x38x64:105x1x64x38
tests:2 passed:0 skipped:0 mistrusted:0 unimplemented:0 invalid_arguments:0 failed:2 listed:0

Pinging @vineelabhinav @Shreyas-fuj from git blame

Environment

  • CPU: Neoverse-V1 C7g.4xlarge
  • OS version: Ubuntu 20.04
  • Compiler version: gcc-10
  • git hash: 319a77eb3da6809476dbf0b512ffacfa88238b76
  • CMake version: 3.16.3
  • CMake output log
+ cmake -DDNNL_CPU_RUNTIME=OMP -DCMAKE_BUILD_TYPE=Release -DDNNL_BUILD_FOR_CI=ON -DDNNL_WERROR=OFF -DDNNL_TEST_SET=NIGHTLY ..
-- The C compiler identification is GNU 10.5.0
-- The CXX compiler identification is GNU 10.5.0
-- Check for working C compiler: /usr/bin/gcc-10
-- Check for working C compiler: /usr/bin/gcc-10 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-10
-- Check for working CXX compiler: /usr/bin/g++-10 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DNNL_TARGET_ARCH: AARCH64
-- DNNL_LIBRARY_NAME: dnnl
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Could NOT find Doxyrest (missing: DOXYREST_EXECUTABLE)
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.18", minimum required is "2.7")
-- Found Sphinx: /usr/bin/sphinx-build (found version "sphinx-build 1.8.5")
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Enabled testing coverage: NIGHTLY
-- Enabled workload: TRAINING
-- Enabled primitives: ALL
-- Enabled primitive CPU ISA: ALL
-- Enabled primitive GPU ISA: ALL
-- Enabled GeMM kernels ISA: ALL
-- Primitive cache is enabled
-- Graph component is enabled
-- Graph compiler backend is disabled.
-- Configuring done
-- Generating done
-- Build files have been written to: oneDNN/build

jondea avatar Jul 24 '24 11:07 jondea