arrayfire-python
arrayfire-python copied to clipboard
arrayfire
JIT issues when using jupyter notebook
Hi, after updating to the latest arrayfire jupyter notebook seems to have a problem with af.gaussian_kernel() function. Standard ipython works without a problem but in notebook the call ends up with:
RuntimeError Traceback (most recent call last)
<ipython-input-3-e9e47878ee5a> in <module>()
----> 1 lala = af.gaussian_kernel(128,128,5,5)
~/anaconda3/lib/python3.6/site-packages/arrayfire/image.py in gaussian_kernel(rows, cols, sigma_r, sigma_c)
778 safe_call(backend.get().af_gaussian_kernel(c_pointer(out.arr),
779 c_int_t(rows), c_int_t(cols),
--> 780 c_double_t(sigma_r), c_double_t(sigma_c)))
781 return out
782
~/anaconda3/lib/python3.6/site-packages/arrayfire/util.py in safe_call(af_error)
77 err_len = c_dim_t(0)
78 backend.get().af_get_last_error(c_pointer(err_str), c_pointer(err_len))
---> 79 raise RuntimeError(to_str(err_str))
80
81 def get_version():
RuntimeError: In function std::vector<char> cuda::compileToPTX(const char*, std::string)
In file src/backend/cuda/jit.cpp:
I tried complete reinstall of python, cleaned any personal settings in .local .bashrc .config and installed completely new anaconda. Still whatever I do I cant get it to run
NOTE: af.constant() function works without problem
SYSTEM: ubuntu 17.10; CUDA 9-2; arrayfire v3.6 binary; arrayfire-python git master
@pavanky I think so, I will reinstall the computer, it will be easier to try with an empty install (at least sys.path and os.environ had same outputs)
Problem still there, fresh ubuntu 18.04, CUDA 9.1, binary af v3.6 (/opt/arrayfire), python from ubuntu repo, arrayfire-python git-master
os.environ diff:
-
ipython3
'_': '/usr/bin/python3' -
jupyter-notebook
'_': '/usr/bin/jupyter-notebook', 'JPY_PARENT_PID': '20126', 'CLICOLOR': '1', 'PAGER': 'cat', 'GIT_PAGER': 'cat', 'MPLBACKEND': 'module://ipykernel.pylab.backend_inline'
sys.path diff:
- jupyter-notebook
'/usr/lib/python3/dist-packages/IPython/extensions', '/home/mk/.ipython'
sys.executable is the same
print(af.info_str())
ArrayFire v3.6.0 (CUDA 64bit)
[0] : GeForce_GTX_1080_Ti (Compute 6.1)
echo $LD_LIBRARY_PATH
/opt/arrayfire/lib:/usr/lib/cuda/lib64:/usr/lib/cuda/nvvm/lib64
echo $AF_PATH
/opt/arrayfire
RuntimeError Traceback (most recent call last)
<ipython-input-3-a1b020a10256> in <module>()
----> 1 af.gaussian_kernel(256,256)
/home/mk/.local/lib/python3.6/site-packages/arrayfire/image.py in gaussian_kernel(rows, cols, sigma_r, sigma_c)
778 safe_call(backend.get().af_gaussian_kernel(c_pointer(out.arr),
779 c_int_t(rows), c_int_t(cols),
--> 780 c_double_t(sigma_r), c_double_t(sigma_c)))
781 return out
782
/home/mk/.local/lib/python3.6/site-packages/arrayfire/util.py in safe_call(af_error)
77 err_len = c_dim_t(0)
78 backend.get().af_get_last_error(c_pointer(err_str), c_pointer(err_len))
---> 79 raise RuntimeError(to_str(err_str))
80
81 def get_version():
RuntimeError: In function std::vector<char> cuda::compileToPTX(const char*, std::string)
In file src/backend/cuda/jit.cpp:
This is weird. Can you run !ldd /opt/arrayfire/lib/libafcuda.so from both locations ?
!ld outputs: (they are the same)
- ipython
!ldd /opt/arrayfire/lib/libafcuda.so linux-vdso.so.1 (0x00007ffd3b3fc000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f44f099a000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f44f0792000) libnvrtc.so.9.1 => /opt/arrayfire/lib/libnvrtc.so.9.1 (0x00007f44ef066000) libcublas.so.9.1 => /opt/arrayfire/lib/libcublas.so.9.1 (0x00007f44eb942000) libcufft.so.9.1 => /opt/arrayfire/lib/libcufft.so.9.1 (0x00007f44e4455000) libcusolver.so.9.1 => /opt/arrayfire/lib/libcusolver.so.9.1 (0x00007f44dece0000) libcusparse.so.9.1 => /opt/arrayfire/lib/libcusparse.so.9.1 (0x00007f44db60c000) libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007f44daa6c000) libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f44da83c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f44da61d000) libfreeimage.so.3 => /usr/lib/x86_64-linux-gnu/libfreeimage.so.3 (0x00007f44da36d000) libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1 (0x00007f44da0e1000) libforge.so.1 => /opt/arrayfire/lib/libforge.so.1 (0x00007f44d9680000) libglbinding.so.2 => /opt/arrayfire/lib/libglbinding.so.2 (0x00007f44d87f1000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f44d8453000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f44d8062000) /lib64/ld-linux-x86-64.so.2 (0x00007f4531c86000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f44d7cd4000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f44d7abc000) libnvidia-fatbinaryloader.so.390.48 => /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.390.48 (0x00007f44d7870000) libjxrglue.so.0 => /usr/lib/x86_64-linux-gnu/libjxrglue.so.0 (0x00007f44d7650000) libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f44d73e8000) libopenjp2.so.7 => /usr/lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007f44d7192000) libpng16.so.16 => /usr/lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f44d6f60000) libraw.so.16 => /usr/lib/x86_64-linux-gnu/libraw.so.16 (0x00007f44d6c8d000) libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f44d6a16000) libwebpmux.so.3 => /usr/lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007f44d680c000) libwebp.so.6 => /usr/lib/x86_64-linux-gnu/libwebp.so.6 (0x00007f44d65a3000) libIlmImf-2_2.so.22 => /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22 (0x00007f44d60e0000) libHalf.so.12 => /usr/lib/x86_64-linux-gnu/libHalf.so.12 (0x00007f44d5e9d000) libIex-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12 (0x00007f44d5c7f000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f44d5a62000) libGLX.so.0 => /usr/lib/x86_64-linux-gnu/libGLX.so.0 (0x00007f44d5831000) libGLdispatch.so.0 => /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007f44d557b000) libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f44d5336000) libGLU.so.1 => /usr/lib/x86_64-linux-gnu/libGLU.so.1 (0x00007f44d50c7000) libjpegxr.so.0 => /usr/lib/x86_64-linux-gnu/libjpegxr.so.0 (0x00007f44d4e93000) liblcms2.so.2 => /usr/lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007f44d4c3b000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f44d4a15000) libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f44d4807000) libIlmThread-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12 (0x00007f44d4600000) libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f44d42c7000) libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f44d4013000) libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f44d3de1000) libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f44d3bb9000) libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f44d39b5000) libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f44d37af000) libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f44d359a000)
- jupyter-notebook
linux-vdso.so.1 (0x00007ffcb4fc7000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3a9d37f000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3a9d177000) libnvrtc.so.9.1 => /opt/arrayfire/lib/libnvrtc.so.9.1 (0x00007f3a9ba4b000) libcublas.so.9.1 => /opt/arrayfire/lib/libcublas.so.9.1 (0x00007f3a98327000) libcufft.so.9.1 => /opt/arrayfire/lib/libcufft.so.9.1 (0x00007f3a90e3a000) libcusolver.so.9.1 => /opt/arrayfire/lib/libcusolver.so.9.1 (0x00007f3a8b6c5000) libcusparse.so.9.1 => /opt/arrayfire/lib/libcusparse.so.9.1 (0x00007f3a87ff1000) libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007f3a87451000) libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f3a87221000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3a87002000) libfreeimage.so.3 => /usr/lib/x86_64-linux-gnu/libfreeimage.so.3 (0x00007f3a86d52000) libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1 (0x00007f3a86ac6000) libforge.so.1 => /opt/arrayfire/lib/libforge.so.1 (0x00007f3a86065000) libglbinding.so.2 => /opt/arrayfire/lib/libglbinding.so.2 (0x00007f3a851d6000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3a84e38000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3a84a47000) /lib64/ld-linux-x86-64.so.2 (0x00007f3ade66b000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3a846b9000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3a844a1000) libnvidia-fatbinaryloader.so.390.48 => /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.390.48 (0x00007f3a84255000) libjxrglue.so.0 => /usr/lib/x86_64-linux-gnu/libjxrglue.so.0 (0x00007f3a84035000) libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f3a83dcd000) libopenjp2.so.7 => /usr/lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007f3a83b77000) libpng16.so.16 => /usr/lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f3a83945000) libraw.so.16 => /usr/lib/x86_64-linux-gnu/libraw.so.16 (0x00007f3a83672000) libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f3a833fb000) libwebpmux.so.3 => /usr/lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007f3a831f1000) libwebp.so.6 => /usr/lib/x86_64-linux-gnu/libwebp.so.6 (0x00007f3a82f88000) libIlmImf-2_2.so.22 => /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22 (0x00007f3a82ac5000) libHalf.so.12 => /usr/lib/x86_64-linux-gnu/libHalf.so.12 (0x00007f3a82882000) libIex-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12 (0x00007f3a82664000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3a82447000) libGLX.so.0 => /usr/lib/x86_64-linux-gnu/libGLX.so.0 (0x00007f3a82216000) libGLdispatch.so.0 => /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007f3a81f60000) libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f3a81d1b000) libGLU.so.1 => /usr/lib/x86_64-linux-gnu/libGLU.so.1 (0x00007f3a81aac000) libjpegxr.so.0 => /usr/lib/x86_64-linux-gnu/libjpegxr.so.0 (0x00007f3a81878000) liblcms2.so.2 => /usr/lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007f3a81620000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f3a813fa000) libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f3a811ec000) libIlmThread-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12 (0x00007f3a80fe5000) libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f3a80cac000) libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f3a809f8000) libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f3a807c6000) libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f3a8059e000) libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f3a8039a000) libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f3a80194000) libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f3a7ff7f000)
From #184, it looks like monte carlo estimation of pi is a good way to reproduce this issue.
This is weird. I was able to reproduce this within ipython (both with regular version and anaconda version).
a = af.randu(5)
af.eval(a * a < 1) # seems to do the trick
I'm having the same issue with JIT when using QtConsole (both from Terminal and inside Spyder). It seems that for some reason ArrayFire loses contact with CUDA toolkit within Jupyter. Because of this, I thought this may be a Jupyter QtConsole issue because ArrayFire connects with CUDA toolkit within Python and iPython consoles opened from Terminal so I'm referencing my issue posted there as well (jupyter/qtconsole#287).
My band-aid at the moment is to run ArrayFire from a terminal Python console. But, if ArrayFire isn't the first package loaded then it will also provide the same error message.
RuntimeError: In function std::vector<char> cuda::compileToPTX(const char*, std::string)
In file src/backend/cuda/jit.cpp:
So, my work around at the moment is as follows:
- Use IDE that runs scripts within a Python console opened from Terminal (MS Visual Code worked out-of-box for me).
- I make sure the first line is
import arrayfire as af. - Then I include
print(af.device_info())to make sure that CUDA toolkit shows up. - Finally, if using MS VSC, 'Run Python File in Terminal' via Right-Click.
I hope this helps anyone in the meantime.
give this thread a bump. It will be very useful if we can use arrayfire in Jupyter notebook environment.
An older version of ArrayFire C library (3.5.1) seems to be working with Jupyter notebook. I would suggest to give it a try.
Is there any big update between 3.5.1 and 3.6.2?
The file can be found in http://arrayfire.s3.amazonaws.com/index.html#!/