ImportError: No module named 'sru_cuda_impl'

[sundiyi]

[Tsinghua-gongjing]

Hi, have you solved the issue? I also met same problem.

[Tsinghua-gongjing]

It seems that Python version affects the result. In my case Python 3.6.6 works fine but 3.7.4 will raise the issue.

[sundiyi]

Python 3.6.6 works fine？ I will try it

[phuntast1c]

For me, the import error was fixed by updating nvcc.

[baijintong]

Can we solve this problem by compiling. cpp file into. so file？

[visionscaper]

I can confirm that downgrading from Python 3.7.4 to Python 3.6.10 mitigates this issue. Why does this happen with Python 3.7?

[visionscaper]

^ I'm very interested for this to be solved, I have other issues when using Python 3.6, and 3.6 is getting a bit old anyway ;-)

[taoleicn]

@visionscaper we've been using Python 3.7.7 without an issue. Could you update the the latest SRU and post the error messages (and stack trace)?

[visionscaper]

I am using the latest code. I'm currently also digging in to this, and it might be related to the fact that I'm working in a virtualenv.

Does it work for you when you do this?:

$ which python3.7
/usr/bin/python3.7

$ virtualenv -p /usr/bin/python3.7 ~/.virtualenvs/sru

# activate the environment
$ source ~/.virtualenvs/sru/bin/activate

(sru) $ pip install sru

# Run you python script using SRU here

(You may need to do sudo pip3 install virtualenv first)

I came to this conclusion because I found this at the beginning of a very long error logging (see the last lines):

Error building extension 'sru_cuda_impl': [1/3] c++ -MMD -MF sru_cuda_impl.o.d -DTORCH_EXTENSION_NAME=sru_cuda_impl -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/TH -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /home/freddy/.virtualenvs/nuhame/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -O3 -c /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/sru/csrc/sru_cuda_impl.cpp -o sru_cuda_impl.o 
FAILED: sru_cuda_impl.o 
c++ -MMD -MF sru_cuda_impl.o.d -DTORCH_EXTENSION_NAME=sru_cuda_impl -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/TH -isystem /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /home/freddy/.virtualenvs/nuhame/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -O3 -c /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/sru/csrc/sru_cuda_impl.cpp -o sru_cuda_impl.o 
In file included from /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/torch/csrc/Device.h:3:0,
                 from /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8,
                 from /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/torch/extension.h:6,
                 from /home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/sru/csrc/sru_cuda_impl.cpp:1:
/home/freddy/.virtualenvs/nuhame/lib/python3.7/site-packages/torch/include/torch/csrc/python_headers.h:10:10: fatal error: Python.h: No such file or directory
 #include <Python.h>
          ^~~~~~~~~~
compilation terminated.

Locating Python.h shows that I have the python3.7-dev installed (which you can get withsudo apt-get install python3.7-dev on Ubuntu:

$ locate Python.h
... multiple entries ...
/usr/include/python3.6m/Python.h
/usr/include/python3.7m/Python.h

So Python.h is installed, but, looking at the error logs I see that -isystem /home/freddy/.virtualenvs/nuhame/include/python3.7m is used instead of /usr/include/python3.7m/, this is likely because I'm in virtualenv nuhame that I'm working from.

I'm not sure yet how I can tell the compilation script to chose /usr/include/python3.7m/, any help is appreciated.

[visionscaper]

@taoleicn As a follow up to my previous comment, I was able to resolve this by adding the path that contains Python.h to the CPATH environment variable, before running my Python script that uses SRU:

export CPATH=$CPATH:/usr/include/python3.7m/

You could add this to your .profile or .bashrc, but that might not be wise.

Now c++ was able to find Python.h, resolving the issue.

Some additional info:

The location of Python.h can be found with locate Python.h; it should be available for the Python version you are using (e.g. 3.7). Further, you can install Python.h by installing the correct Python dev package, e.g:

sudo apt-get install python3.7-dev

This is only an issue when you work from an Python virtual environment as indicated earlier.