silx CI tests fails on gh actions with macos due to OpenCL error

trafficstars

CI currently fails on gh actions with macos. I restarted a PR that was fine a few days ago and it is now broken.

See e.g., https://github.com/silx-kit/silx/pull/3455/checks?check_run_id=2527669607 -> Run the tests

INFO:silx.opencl.common:For Apple's OpenCL on CPU: Measuring actual valid max_work_goup_size.
Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/tools.py", line 100, in wrapper
    return ctx_dict[cl_object][cache_key]
KeyError: <pyopencl.Context at 0x7f82b4e94620 on <pyopencl.Device 'Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz' on 'Apple' at 0xffffffff>>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_tests.py", line 455, in <module>
    unittest.defaultTestLoader.loadTestsFromNames(options.test_name))
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/unittest/loader.py", line 220, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/unittest/loader.py", line 220, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/unittest/loader.py", line 205, in loadTestsFromName
    test = obj()
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/test/__init__.py", line 69, in suite
    from ..image import test as test_image
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/image/test/__init__.py", line 33, in <module>
    from . import test_medianfilter
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/image/test/test_medianfilter.py", line 34, in <module>
    from silx.image import medianfilter
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/image/medianfilter.py", line 38, in <module>
    from silx.opencl import ocl as _ocl
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/opencl/__init__.py", line 52, in <module>
    from .common import *
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/opencl/common.py", line 301, in <module>
    class OpenCL(object):
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/opencl/common.py", line 354, in OpenCL
    workgroup = _measure_workgroup_size(device, fast=True)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/silx/opencl/common.py", line 266, in _measure_workgroup_size
    d_data_1.fill(numpy.float32(1.0))
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/array.py", line 1332, in fill
    self._fill(self, value, queue=queue, wait_for=wait_for))
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/array.py", line 176, in kernel_runner
    knl = kernel_getter(*args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/array.py", line 889, in _fill
    return elementwise.get_fill_kernel(result.context, result.dtype)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/tools.py", line 103, in wrapper
    result = func(cl_object, *args, **kwargs)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/elementwise.py", line 702, in get_fill_kernel
    return get_elwise_kernel(context,
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/elementwise.py", line 172, in get_elwise_kernel
    func, arguments = get_elwise_kernel_and_types(
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/elementwise.py", line 154, in get_elwise_kernel_and_types
    prg = get_elwise_program(
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/elementwise.py", line 107, in get_elwise_program
    return Program(context, source).build(options)
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/__init__.py", line 531, in build
    self._prg, was_cached = self._build_and_catch_errors(
  File "/Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/__init__.py", line 579, in _build_and_catch_errors
    raise err
pyopencl._cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz' on 'Apple' at 0xffffffff>:


(options: -I /Users/runner/hostedtoolcache/Python/3.8.9/x64/lib/python3.8/site-packages/pyopencl/cl)
(source saved as /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp42w8b0bx.cl)

May 07 '21 14:05 t20100

Andreas Kloekner modified the cache of built program recently ... I'll investigate on Monday, probably

May 07 '21 14:05 kif

OK, thanks! There was no recent release of pyopencl (the CI of some recent PR passed with the same version as the one that no fails).

May 07 '21 14:05 t20100

still trying to configure macos to get the connection to the network

May 07 '21 15:05 kif

I confirm I can reproduce the bug ... neither the CPU nor the GPU works anymore on MacOS

May 07 '21 16:05 kif

OpenCL stopped working on my mac ... this is not limited to pyopencl. clpeak neither works

May 07 '21 20:05 kif

I believe, the only thing that remains to be done is remove OpenCL from MacOS test-suite.

Does it still work on your mac, Thomas ? I had terrible problems with the latest "security" update from Apple for Catalina: both my USB3-network adapter and the OpenVPN client have seen the signature of their kernel extension dropped and stopped working (i.e. it lost all network connections). Rotten Apple !

May 07 '21 21:05 kif

I'm still using 10.12

May 10 '21 08:05 t20100

On Mon, 10 May 2021 01:25:24 -0700 Thomas VINCENT @.***> wrote:

I'm still using 10.12 does it work on your computer ?

May 10 '21 12:05 kif

With current tip of the project, OpenCL tests run (python run_tests.py silx.opencl.test.suite) with 2 errors:

======================================================================
ERROR: test_laplacian (silx.opencl.test.test_linalg.TestLinAlg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "silx/opencl/test/test_linalg.py", line 200, in test_laplacian
    laplacian_ref = laplace(self.image)
  File "lib/python3.7/site-packages/scipy/ndimage/filters.py", line 436, in laplace
    return generic_laplace(input, derivative2, output, mode, cval)
  File "lib/python3.7/site-packages/scipy/ndimage/filters.py", line 402, in generic_laplace
    *extra_arguments, **extra_keywords)
  File "lib/python3.7/site-packages/scipy/ndimage/filters.py", line 435, in derivative2
    return correlate1d(input, [1, -2, 1], axis, output, mode, cval, 0)
  File "lib/python3.7/site-packages/scipy/ndimage/filters.py", line 80, in correlate1d
    output = _ni_support._get_output(output, input)
  File "lib/python3.7/site-packages/scipy/ndimage/_ni_support.py", line 82, in _get_output
    raise RuntimeError("output shape not correct")
RuntimeError: output shape not correct

======================================================================
ERROR: test_matching (silx.opencl.sift.test.test_matching.TestMatching)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "silx/opencl/sift/test/test_matching.py", line 164, in test_matching
    delta = abs(res_sort - siftmatch).max()
ValueError: operands could not be broadcast together with shapes (496,2) (480,2) 

======================================================================
SKIPPED: test_1D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_batched_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_1D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_batched_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_1D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_batched_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_1D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_batched_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_nonseparable_3D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_2D (silx.opencl.test.test_convolution.TestConvolution)
SKIPPED: test_separable_3D (silx.opencl.test.test_convolution.TestConvolution)
----------------------------------------------------------------------
mode=constant not implemented without textures
----------------------------------------------------------------------
Ran 254 tests in 41.365s

FAILED (errors=2, skipped=24)

May 10 '21 13:05 t20100

Closing, reopen if needed

Sep 08 '22 14:09 t20100

silx silx copied to clipboard

CI tests fails on gh actions with macos due to OpenCL error

silx
silx copied to clipboard