Caching problem with building on CPU MacOSX Mojave ?

[yves-surrel]

Since I have update to Mojave, I get building failures on CPU device.

If I run

import pyopencl as cl

a_np = np.random.rand(50000).astype(np.float32)
b_np = np.random.rand(50000).astype(np.float32)

ctx = cl.create_some_context()
queue = cl.CommandQueue(ctx)

mf = cl.mem_flags
a_g = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=a_np)
b_g = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=b_np)

prg = cl.Program(ctx, """
__kernel void sum(
    __global const float *a_g, __global const float *b_g, __global float *res_g)
{
  int gid = get_global_id(0);
  res_g[gid] = a_g[gid] + b_g[gid];
}
""")

prg=prg.build()

res_g = cl.Buffer(ctx, mf.WRITE_ONLY, a_np.nbytes)
prg.sum(queue, a_np.shape, None, a_g, b_g, res_g)

res_np = np.empty_like(a_np)
cl.enqueue_copy(queue, res_np, res_g)

# Check on CPU with Numpy:
print(res_np - (a_np + b_np))
print(np.linalg.norm(res_np - (a_np + b_np)))

it works only once. Every attempt to build the same Program again will raise

RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE - clBuildProgram failed: BUILD_PROGRAM_FAILURE

Build on <pyopencl.Device 'Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz' on 'Apple' at 0xffffffff>:

If I change the kernel source, e.g. inserting a blank line or a comment, the build will succeed but only once. I suspect it has something to do with the caching mechanism somewhere.

Apparently related to #252, because when I run for the first time the code there, it succeeded, but now I always get the build failure.

EDIT If some build option is added e.g.

prg=prg.build(options=['-cl-opt-disable'])

the build will succeed once, and then will fail again. If the option is changed, it will succeed once then fail, etc.

[inducer]

Could you try and delete the compiler caches and report back on whether that fixes things? ($HOME/.cache/pytools/pdict-v2-pyopencl-*) It may just be that you have some old binaries from pre-Mojave sitting around, and the Mojave ICDs no longer like those. If that's the case, we just need to make sure we pick up on that when generating the cache key.

[isuruf]

This seems to be an issue with the opencl implementation in 10.14. https://github.com/magnumripper/JohnTheRipper/issues/3434. I can reproduce on a fresh VM on travis-ci OSX

[yves-surrel]

@inducer: I have not this directory (I am using Enthought Canopy, so the directory structure may be different). Is there any command to get the actual cache directory? However, I agree with isuruf, it seems to be likely to be a Mojave issue. But it looks like removing some file somewhere should fix the problem.

[inducer]

@isuruf Thanks for investigating. Setting the environment variable PYOPENCL_NO_CACHE=1 should disable binary caching and might help work around this problem. Does Apple not have quality control?

[yves-surrel]

@inducer: yes, it fixes the problem. Thx a lot