pyFAI icon indicating copy to clipboard operation
pyFAI copied to clipboard
verified publisher icon silx-kit

Numerical stability issue when running tests on i386: use numpy.allclose

Open kif opened this issue 6 years ago • 1 comments 

======================================================================
FAIL: test_split_bbox_2d (pyFAI.test.test_split_pixel.TestSplitBBoxNg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/picca/pyFAI/build/lib.linux-i686-3.7/pyFAI/test/test_split_pixel.py", line 205, in test_split_bbox_2d
    self.assertEqual(abs(count_legacy - count_ng).max(), 0, "count is the same")
AssertionError: 9.237055564881302e-14 != 0 : count is the same

kif avatar Jan 23 '19 19:01 kif

We can expect that the default tolerance from all_close are good.

kif avatar Jan 23 '19 20:01 kif

Debian has recently reported another bug with most 32bits architectures. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1018811

Tested a 32-bit chroot of debian sid on i386, the integrated test-suite of the released version fails on:

======================================================================
FAIL: test_count_csr (pyFAI.test.test_histogram.TestHistogram2d)
Test that the pixel count and the total intensity is conserved
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/test/test_histogram.py", line 339, in test_count_csr
    self.assertTrue(delta == 0, msg="check all pixels were counted")
AssertionError: False is not true : check all pixels were counted

======================================================================
FAIL: test_numpy_vs_cython_vs_csr_2d (pyFAI.test.test_histogram.TestHistogram2d)
Compare numpy histogram with cython simple implementation
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/test/test_histogram.py", line 373, in test_numpy_vs_cython_vs_csr_2d
    self.assertTrue(delta_max <= self.err_max_cnt, "pixel count difference numpy/csr : max delta=%s" % delta_max)
AssertionError: False is not true : pixel count difference numpy/csr : max delta=8.0

======================================================================
FAIL: test_2d_nosplit (pyFAI.test.test_csr.TestCSR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/test/test_csr.py", line 195, in test_2d_nosplit
    self.assertLess(error.mean(), 1e-3, "img are almost the same")
AssertionError: 244.15215998872887 not less than 0.001 : img are almost the same

======================================================================
FAIL: test_dw_div_dw (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 699, in test_dw_div_dw
    self.assertLess(abs(self.a / self.b - res).max(), 6 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1342405481507001e-07 not less than 8.526512829121202e-14 : Exact matches

======================================================================
FAIL: test_dw_div_fp (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 681, in test_dw_div_fp
    self.assertLess(abs(self.a / self.bh - res).max(), 3 * EPS32 ** 2, "Exact matches")
AssertionError: 5.953229997857079e-08 not less than 4.263256414560601e-14 : Exact matches

======================================================================
FAIL: test_dw_plus_dw (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 629, in test_dw_plus_dw
    self.assertLess(abs(self.a + self.b - res).max(), 3 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1911641628614689e-07 not less than 4.263256414560601e-14 : Exact matches

======================================================================
FAIL: test_dw_plus_fp (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 611, in test_dw_plus_fp
    self.assertLess(abs(self.ah.astype(numpy.float64) + self.al + self.bh - res).max(), 2 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1896441831638072e-07 not less than 2.842170943040401e-14 : Exact matches

======================================================================
FAIL: test_dw_times_dw (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 664, in test_dw_times_dw
    self.assertLess(abs(self.a * self.b - res).max(), 5 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1907044639158926e-07 not less than 7.105427357601002e-14 : Exact matches

======================================================================
FAIL: test_dw_times_fp (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 646, in test_dw_times_fp
    self.assertLess(abs(self.a * self.bh - res).max(), 2 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1908770014557035e-07 not less than 2.842170943040401e-14 : Exact matches

======================================================================
FAIL: test_fast_sum2 (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 564, in test_fast_sum2
    self.assertEqual(abs(self.ah.astype(numpy.float64) + self.bl - (res_h.get().astype(numpy.float64) + res_l.get())).max(), 0, "Exact matches")
AssertionError: 5.953905457545261e-08 != 0 : Exact matches

======================================================================
FAIL: test_sum2 (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 578, in test_sum2
    self.assertEqual(abs(self.ah.astype(numpy.float64) + self.bh - (res_h.get().astype(numpy.float64) + res_l.get())).max(), 0, "Exact matches")
AssertionError: 1.1920928955078125e-07 != 0 : Exact matches

======================================================================
FAIL: test_integrate_ng (pyFAI.opencl.test.test_ocl_azim_csr.TestOclAzimCSR)
tests the 1d histogram kernel, default block size
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_ocl_azim_csr.py", line 138, in test_integrate_ng
    self.integrate_ng(block_size=None)
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_ocl_azim_csr.py", line 115, in integrate_ng
    self.assertLessEqual(delta.max(), 1, "counts are almost the same")
AssertionError: 5.0 not less than or equal to 1 : counts are almost the same

======================================================================
FAIL: test_integrate_ng_single (pyFAI.opencl.test.test_ocl_azim_csr.TestOclAzimCSR)
tests the 1d histogram kernel, default block size
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_ocl_azim_csr.py", line 145, in test_integrate_ng_single
    self.integrate_ng(block_size=1)
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_ocl_azim_csr.py", line 115, in integrate_ng
    self.assertLessEqual(delta.max(), 1, "counts are almost the same")
AssertionError: 20.0 not less than or equal to 1 : counts are almost the same

======================================================================
FAIL: test_integrate_ng (pyFAI.opencl.test.test_ocl_azim_lut.TestOclAzimLUT)
tests the 1d histogram kernel
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_ocl_azim_lut.py", line 116, in test_integrate_ng
    self.assertLessEqual(delta.max(), 1, "counts are almost the same")
AssertionError: 36.0 not less than or equal to 1 : counts are almost the same

kif avatar Dec 09 '22 08:12 kif

The same test on the current development branch fails at:

======================================================================
FAIL: test_dw_div_dw (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 699, in test_dw_div_dw
    self.assertLess(abs(self.a / self.b - res).max(), 6 * EPS32 ** 2, "Exact matches")
AssertionError: 1.0914740578016335e-07 not less than 8.526512829121202e-14 : Exact matches

======================================================================
FAIL: test_dw_div_fp (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 681, in test_dw_div_fp
    self.assertLess(abs(self.a / self.bh - res).max(), 3 * EPS32 ** 2, "Exact matches")
AssertionError: 5.936286551211367e-08 not less than 4.263256414560601e-14 : Exact matches

======================================================================
FAIL: test_dw_plus_dw (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 629, in test_dw_plus_dw
    self.assertLess(abs(self.a + self.b - res).max(), 3 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1905879127027674e-07 not less than 4.263256414560601e-14 : Exact matches

======================================================================
FAIL: test_dw_plus_fp (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 611, in test_dw_plus_fp
    self.assertLess(abs(self.ah.astype(numpy.float64) + self.al + self.bh - res).max(), 2 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1895980822629326e-07 not less than 2.842170943040401e-14 : Exact matches

======================================================================
FAIL: test_dw_times_dw (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 664, in test_dw_times_dw
    self.assertLess(abs(self.a * self.b - res).max(), 5 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1897606588817666e-07 not less than 7.105427357601002e-14 : Exact matches

======================================================================
FAIL: test_dw_times_fp (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 646, in test_dw_times_fp
    self.assertLess(abs(self.a * self.bh - res).max(), 2 * EPS32 ** 2, "Exact matches")
AssertionError: 1.1909269082011065e-07 not less than 2.842170943040401e-14 : Exact matches

======================================================================
FAIL: test_fast_sum2 (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 564, in test_fast_sum2
    self.assertEqual(abs(self.ah.astype(numpy.float64) + self.bl - (res_h.get().astype(numpy.float64) + res_l.get())).max(), 0, "Exact matches")
AssertionError: 5.957427617886424e-08 != 0 : Exact matches

======================================================================
FAIL: test_sum2 (pyFAI.opencl.test.test_openCL.TestDoubleWord)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyFAI/opencl/test/test_openCL.py", line 578, in test_sum2
    self.assertEqual(abs(self.ah.astype(numpy.float64) + self.bh - (res_h.get().astype(numpy.float64) + res_l.get())).max(), 0, "Exact matches")
AssertionError: 1.1920928955078125e-07 != 0 : Exact matches

======================================================================

All tests related to histograms have been fixed, which is great. Remains the tests on numerical error compensation mechanism which is biased on i386 since it uses x87 registers of 80bits instead of 64bits with SSE2. I'll open another issue on this

kif avatar Dec 09 '22 08:12 kif