No module named 'cuda_functional'

[wangye8899]

[Lkn1995]

hi, i met the same error, have you solve it

[sundiyi]

i also met the same error, have you solve it

[taoleicn]

hi can you share the stack trace with the error messages? also, the new version >= 2.1.8 requires ninja to compile the CUDA code. please make sure you have it installed.

[sundiyi]

C:\ProgramData\Anaconda3\python.exe C:/孙鹏飞/sru/sru-master/sru-master/DYNAMIC/main2.py C:\ProgramData\Anaconda3\lib\site-packages\torch\nn_reduction.py:43: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead. warnings.warn(warning.format(ret)) C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py:189: UserWarning: Error checking compiler version for cl: [WinError 2] 系统找不到指定的文件。 warnings.warn('Error checking compiler version for {}: {}'.format(compiler, error)) ��Ϣ: ���ṩ��ģʽ�޷��ҵ��ļ��� Traceback (most recent call last): File "C:\孙鹏飞\sru\sru-master\sru-master\DYNAMIC\sru\sru_functional.py", line 35, in _lazy_load_cuda_kernel from .cuda_functional import SRU_Compute_GPU File "C:\孙鹏飞\sru\sru-master\sru-master\DYNAMIC\sru\cuda_functional.py", line 14, in verbose=False File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 658, in load is_python_module) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 827, in _jit_compile with_cuda=with_cuda) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 876, in _write_ninja_file_and_build with_cuda=with_cuda) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 1089, in _write_ninja_file 'cl']).decode().split('\r\n') File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 336, in check_output **kwargs).stdout File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['where', 'cl']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:/孙鹏飞/sru/sru-master/sru-master/DYNAMIC/main2.py", line 121, in output,hidden = encoder(x,h0) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "C:/孙鹏飞/sru/sru-master/sru-master/DYNAMIC/main2.py", line 37, in forward output, hidden = self.sru(input, hidden) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "C:\孙鹏飞\sru\sru-master\sru-master\DYNAMIC\sru\sru_functional.py", line 564, in forward h, c = rnn(prevx, c0[i], mask_pad=mask_pad) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "C:\孙鹏飞\sru\sru-master\sru-master\DYNAMIC\sru\sru_functional.py", line 363, in forward SRU_Compute = _lazy_load_cuda_kernel()( File "C:\孙鹏飞\sru\sru-master\sru-master\DYNAMIC\sru\sru_functional.py", line 37, in _lazy_load_cuda_kernel from cuda_functional import SRU_Compute_GPU ModuleNotFoundError: No module named 'cuda_functional' i am sure that i have install ninja

[hadaev8]

Same here, win10

[lanisyutin]

Got the same error trying to execute example from Examples section. It raises only on latest version (2.1.9)

[fann1993814]

Hi, guys~ I solved this problem. That's a odd phenomenon in windows. The SRU is compiled when it is executed first time, so we need to set the MSVC in windows.

My environment cuda 10.1 python 3.6.8 pytorch 1.3.1 sru 2.1.9 ninja 1.9.0

First. install MSVC 2017 (include VC++ 2017 version 15.9 v14.16 latest v141 tools and Windows 10 SDK), then execute the script "VS 2017 x64 Native Tools" and run your model code. The SRU is generated into several files below the folder ("C:\Users{your name}\AppData\Local\Temp\torch_extensions\sru_cuda_impl").

[hadaev8]

Well same on linux https://colab.research.google.com/drive/1Sk-XoxRcym9izg9wsnmKnAFL1CWXkzVL

[fann1993814]

Well same on linux https://colab.research.google.com/drive/1Sk-XoxRcym9izg9wsnmKnAFL1CWXkzVL

Maybe check compiler setting? apt-get install build-essential ?

there is no any problem on my linux server (ubuntu and MacOS).

[hadaev8]

I have run it again with same notebook and now it runs without problems. Strange.

[hadaev8]

@fann1993814 I tring to run on amazon linux, but kernel crush without warnings, any ideas?

[sundiyi]

Hi, guys~ I solved this problem. That's a odd phenomenon in windows. The SRU is compiled when it is executed first time, so we need to set the MSVC in windows.

My environment cuda 10.1 python 3.6.8 pytorch 1.3.1 sru 2.1.9 ninja 1.9.0

First. install MSVC 2017 (include VC++ 2017 version 15.9 v14.16 latest v141 tools and Windows 10 SDK), then execute the script "VS 2017 x64 Native Tools" and run your model code. The SRU is generated into several files below the folder ("C:\Users{your name}\AppData\Local\Temp\torch_extensions\sru_cuda_impl").

C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py:189: UserWarning: Error checking compiler version for cl: [WinError 2] 系统找不到指定的文件。 Emitting ninja build file C:\Users\飞\AppData\Local\Temp\torch_extensions\sru_cuda_impl\build.ninja... warnings.warn('Error checking compiler version for {}: {}'.format(compiler, error)) ��Ϣ: ���ṩ��ģʽ�޷��ҵ��ļ��� Using C:\Users\飞\AppData\Local\Temp\torch_extensions as PyTorch extensions root... No modifications detected for re-loaded extension module sru_cuda_impl, skipping build step... Loading extension module sru_cuda_impl... Traceback (most recent call last): File "C:\ProgramData\Anaconda3\lib\site-packages\sru\sru_functional.py", line 35, in _lazy_load_cuda_kernel from .cuda_functional import SRU_Compute_GPU File "C:\ProgramData\Anaconda3\lib\site-packages\sru\cuda_functional.py", line 15, in verbose=True File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 658, in load is_python_module) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 827, in _jit_compile with_cuda=with_cuda) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 876, in _write_ninja_file_and_build with_cuda=with_cuda) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 1089, in _write_ninja_file 'cl']).decode().split('\r\n') File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 395, in check_output **kwargs).stdout File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 487, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['where', 'cl']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "E:/sru-master/sru-master/misc/compare_gpu_speed_sru_gru.py.py", line 83, in benchmark_sru_gpu() File "E:/sru-master/sru-master/misc/compare_gpu_speed_sru_gru.py.py", line 66, in benchmark_sru_gpu output, hn = rnn(input, h0) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "C:\ProgramData\Anaconda3\lib\site-packages\sru\sru_functional.py", line 564, in forward h, c = rnn(prevx, c0[i], mask_pad=mask_pad) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "C:\ProgramData\Anaconda3\lib\site-packages\sru\sru_functional.py", line 363, in forward SRU_Compute = _lazy_load_cuda_kernel()( File "C:\ProgramData\Anaconda3\lib\site-packages\sru\sru_functional.py", line 37, in _lazy_load_cuda_kernel from .cuda_functional import SRU_Compute_GPU File "C:\ProgramData\Anaconda3\lib\site-packages\sru\cuda_functional.py", line 15, in verbose=True File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 658, in load is_python_module) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 838, in _jit_compile return _import_module_from_library(name, build_directory, is_python_module) File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py", line 978, in _import_module_from_library file, path, description = imp.find_module(module_name, [path]) File "C:\ProgramData\Anaconda3\lib\imp.py", line 296, in find_module raise ImportError(_ERR_MSG.format(name), name=name) ImportError: No module named 'sru_cuda_impl'

hi, the problem issolved? could tell me how to solve it on Win10

[maxbaluev]

@taolei87 @taoleicn @hpasapp Any ideas? I think the bug affects all the operating systems.

[fann1993814]

Download this file (MSVC Build Tool for 2017, Don't use 2019, it will failed) https://aka.ms/vs/15/release/vs_buildtools.exe

And Install items (like this, sorry my language is Chinese)

Finally, add (C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64) into your OS Environment Variable PATH

Follow steps

1. Execute the script "VS 2017 x64 Native Tools"
2. Run example code (or your code) in the Command Prompt (python xxx.py)
3. Waiting for compiling (Please confirm your memory is higher than 4G)

It won't compile again, so the second time execution is very fast.

My Env: Windows 10 CUDA 10.1 Pytorch 1.3 Python 3.6.8

[snsxf]

same problem on Linux, how to solve it?

[hpasapp]

@snsxf on linux, check you have the version of cupy corresponding to your cuda toolkit installed. eg if you have CUDA toolkit 10.1, check you have installed cupy-cuda101.

[hadaev8]

@fann1993814 I have feeling it compile every time on windows. It work only from VS 2017 x64 Native Tools and every time i run it it write

E:\neuro_projects\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py:209: UserWarning: Error checking compiler version for cl: 'utf-8' codec can't decode byte 0x8e in position 0: invalid start byte warnings.warn('Error checking compiler version for {}: {}'.format(compiler, error))

[fanfoxle]

Download this file (MSVC Build Tool for 2017, Don't use 2019, it will failed) https://aka.ms/vs/15/release/vs_buildtools.exe

And Install items (like this, sorry my language is Chinese)

Finally, add (C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64) into your OS Environment Variable PATH

Follow steps

1. Execute the script "VS 2017 x64 Native Tools"
2. Run example code (or your code) in the Command Prompt (python xxx.py)
3. Waiting for compiling (Please confirm your memory is higher than 4G)

It won't compile again, so the second time execution is very fast.

My Env: Windows 10 CUDA 10.1 Pytorch 1.3 Python 3.6.8

Thx, this works. But the code must be implemented from x64 native tools command prompt. If I run it in Windows shell or directly from VS code, the error "ModuleNotFoundError: No module named 'cuda_functional'" is still shown.

[dataprowl]

@hpasapp The version got from nvcc --version differs from nvidia-smi. However, I have tried to install cupy for both the versions in separate conda environments, but I still get the error ModuleNotFoundError: No module named 'cuda_functional'. In both cases.

Also I have followed the comment here by @taolei87 . With no luck.

My Env: Ubuntu 18.04 CUDA 10.2 Pytorch 1.4 Python 3.7

[hpasapp]

I would say that having.a different CUDA version reported by nvidia-smi compared to the version reported by nvcc --version sounds odd to me. I think you should try to figure out why they are different, and ensure they are the same (they are the same on my system: both 10.1).

[dataprowl]

@hpasapp That's perfectly right! I purged the Nvidia driver and Cuda toolkit, and installed them again. It works like charm. Thank you.

[hpasapp]

Great!

On Fri, Apr 3, 2020 at 8:33 AM dataprowl [email protected] wrote:

@hpasapp https://github.com/hpasapp That's perfectly right! I purged the Nvidia driver and Cuda toolkit, and installed them again. It works like charm. Thank you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/asappresearch/sru/issues/109#issuecomment-608408126, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEZ5OBGY3I7OI5LJDXPVFGLRKXJS3ANCNFSM4JKADMMA .

-- HUGH PERKINS Research Engineer

https://www.asapp.com/ One World Trade Center 80th Floor New York, NY 10007

[SSS135]

@hadaev8

I have feeling it compile every time on windows. It work only from VS 2017 x64 Native Tools and every time i run it it write

E:\neuro_projects\Anaconda3\lib\site-packages\torch\utils\cpp_extension.py:209: UserWarning: Error checking compiler version for cl: 'utf-8' codec can't decode byte 0x8e in position 0: invalid start byte warnings.warn('Error checking compiler version for {}: {}'.format(compiler, error))

I've had same problem but found a dirty fix:

1. Clean build folder (c:\Users\username\AppData\Local\Temp\torch_extensions\sru_cuda_impl)
2. Run SRU example NOT from VS 2017 x64 Native Tools
3. In VS Tools switch working directory to build folder and run "ninja -v"

Not sure why it helped, maybe because VS Tools use its own ninja version in "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe"

[xelibrion]

Once I fiddled with the code of _lazy_load_cuda_kernel, so it looks like this:

# load C++ implementation for GPU computation
def _lazy_load_cuda_kernel():
    # try:
    from sru.cuda_functional import SRU_Compute_GPU

    # except:
    # from cuda_functional import SRU_Compute_GPU
    return SRU_Compute_GPU

, I got a way more meaningful message from ninja, stating it does not support version of GCC installed on my machine:

Traceback (most recent call last):
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 949, in _build_extension_module
    check=True)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "train.py", line 277, in <module>
    train(args)
  File "train.py", line 100, in train
    pred = model(query, support, support_label)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xelibrion/projects/dynamic-classification/model.py", line 333, in forward
    query_encoding = self.encoder(query_embeddings, padding_mask=padding_mask)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xelibrion/projects/dynamic-classification/model.py", line 175, in forward
    output, state = self.rnn(data, state, mask_pad=(-padding_mask + 1).byte())
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/sru/sru_functional.py", line 595, in forward
    h, c = rnn(prevx, c0[i], mask_pad=mask_pad)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/sru/sru_functional.py", line 386, in forward
    SRU_Compute = _lazy_load_cuda_kernel()(
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/sru/sru_functional.py", line 37, in _lazy_load_cuda_kernel
    from sru.cuda_functional import SRU_Compute_GPU
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/sru/cuda_functional.py", line 14, in <module>
    verbose=False
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 644, in load
    is_python_module)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 813, in _jit_compile
    with_cuda=with_cuda)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 866, in _write_ninja_file_and_build
    _build_extension_module(name, build_directory, verbose)
  File "/home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 962, in _build_extension_module
    raise RuntimeError(message)
RuntimeError: Error building extension 'sru_cuda_impl': b"[1/2] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=sru_cuda_impl -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include/TH -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -std=c++11 -c /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/sru/sru_cuda_kernel.cu -o sru_cuda_kernel.cuda.o\nFAILED: sru_cuda_kernel.cuda.o \n/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=sru_cuda_impl -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include/TH -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /home/xelibrion/anaconda3/envs/dialog-intent-induction/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -std=c++11 -c /home/xelibrion/anaconda3/envs/dialog-intent-induction/lib/python3.7/site-packages/sru/sru_cuda_kernel.cu -o sru_cuda_kernel.cuda.o\nIn file included from /opt/cuda/include/cuda_runtime.h:83,\n                 from <command-line>:\n/opt/cuda/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported!\n  138 | #error -- unsupported GNU version! gcc versions later than 8 are not supported!\n      |  ^~~~~\nninja: build stopped: subcommand failed.\n"

[bratao]

Just to comment about it, I had the same error, but it was masked because of the try catch. In my case, I was missing the nvcc . Changing the docker image to nvidia/cuda:10.2-cudnn7-devel solved it

[zb-tjw]

同样遇到这个问题，自己的解决过程如下：https://blog.csdn.net/qq_32239767/article/details/109626626