scikit-cuda icon indicating copy to clipboard operation
scikit-cuda copied to clipboard

Suggestion: adding a last-ditch ctypes.util.find_library before raising OSError

Open johnh2o2 opened this issue 6 years ago • 2 comments

I'm using skcuda in my own library and trying to run sphinx with autodoc, but anytime autodoc attempts to load a CUDA-related .dylib (e.g. libcufft.dylib) ctypes doesn't find the file. It's very strange, as running outside of autodoc does not produce this problem, but if I use ctypes.util.find_library, ctypes can find the absolute path and there are no further problems.

If it's not too much trouble, would it be a problem to replace, for example in the cublas.py file:

if _libcublas == None:
    raise OSError('cublas library not found')

with

if _libcublas == None:
    try:
        _libcublas = ctypes.cdll.LoadLibrary(ctypes.util.find_library(_libcublas_libname))
    except OSError:
        raise OSError('cublas library not found')

or is there a reason not to do this? If so I can make a pull request and add in the changes, but I wanted to check here. Maybe there's a better solution? I'm not very familiar with how ctypes is trying to search for libraries, and I am completely stumped as to why this is only a problem if run with autodoc...

johnh2o2 avatar Sep 26 '17 19:09 johnh2o2

Here are some more details about my configuration in case that's useful, but my suggestion is independent of the problem:

  • running Python 2.7.13 (Anaconda 2).
  • sphinx-build 1.6.3
  • Macbook pro, Sierra 10.12.5 (16F73)
(pycu) [jah5@macpro:cuvarbase]$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:46_CST_2017
Cuda compilation tools, release 8.0, V8.0.61
  • The nvidia runtime libraries are installed in /usr/local/cuda/lib but libraries are symlinked to their location on /Developer/NVIDIA/CUDA-8.0/lib/

Anyway, let me know if there's any more information you want about the problem itself, but the ctypes.util.find_library was enough to solve whatever was causing the issue and I can't think of a good reason not to perform that check as an additional backup measure.

johnh2o2 avatar Sep 27 '17 13:09 johnh2o2

You can try mocking up the skcuda modules imported by your package in your sphinx configuration so that the import error doesn't break autodoc; see the docs/source/conf.py for how I did this to enable skcuda's docs to build on Read The Docs. (Sphinx does have an option to mock specified dependencies, but it might not work for modules that define attribs accessed by the package that imports them; you might want to give it a try before resorting to the approach in skcuda's docs.)

lebedov avatar Nov 23 '17 18:11 lebedov