from scikits.odes.sundials.cvode cimport CV_RhsFunction does not work

[cmashraf]

Following up with issue https://github.com/bmcage/odes/issues/107, when I am using the command

from scikits.odes.sundials.cvode cimport CV_RhsFunction

and cythonize, I get the following error

Error compiling Cython file:

... print("Hello") import numpy as np from cpython cimport bool cimport numpy as np from scikits.odes.sundials.cvode cimport CV_RhsFunction ^

hello.pyx:5:0: 'scikits/odes/sundials/cvode.pxd' not found

Error compiling Cython file:

... print("Hello") import numpy as np from cpython cimport bool cimport numpy as np from scikits.odes.sundials.cvode cimport CV_RhsFunction ^

hello.pyx:5:0: 'scikits/odes/sundials/cvode/CV_RhsFunction.pxd' not found Traceback (most recent call last): File "setup.py", line 5, in ext_modules=cythonize("hello.pyx")) File "/gscratch/pfaendtner/Chowdhury/codes/Anaconda/envs/odes/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1096, in cythonize cythonize_one(*args) File "/gscratch/pfaendtner/Chowdhury/codes/Anaconda/envs/odes/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1219, in cythonize_one raise CompileError(None, pyx_file) Cython.Compiler.Errors.CompileError: hello.pyx

I get the same error too when I try to use cython magic in Jupiter notebook. I think I managed to install both the sundials and scikits.odes successfully as I can run other non-cython examples and nosetests. I also have set SUNDIALS_INST and LD_LIBRARY_PATH.

Any idea why this is not working?

[utsekaj42]

What command are you running? I have tried the following based on 70 but without success.

cython  -I ~/anaconda3/envs/skodes/lib/python3.7/site-packages/scikits/odes/sundials/  \
            -I ~/odes/scikits/odes/sundials/ \
            -I ~/anaconda3/envs/skodes/include/cvode/ \
            van_der_pol_fun.pyx


Error compiling Cython file:
------------------------------------------------------------
...
"""
from __future__ import division
import numpy as np
cimport numpy as np
cimport scikits.odes.sundials.cvode
       ^
------------------------------------------------------------

van_der_pol_fun.pyx:37:8: 'scikits/odes/sundials/cvode.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
ctypedef np.float_t DTYPE_t

cdef double mu = 1000;

# Right-hand side function
cdef class CV_Rhs_van_der_pol_cy(CV_RhsFunction):
    ^
------------------------------------------------------------

van_der_pol_fun.pyx:45:5: 'CV_RhsFunction' is not a type name

I have tried variations of how I import CV_RhsFunction.

If I use the example as given from scikits.odes.sundials.cvode cimport CV_RhsFunction, the error message says it cannot find CV_RhsFunction.pxd, which makes sense as this file doesn't exist. However

cimport scikits.odes.sundials.cvode
CV_RhsFunction = scikits.odes.sundials.cvode.CV_RhsFunction

results in the above output, but both ~/odes/scikits/odes/sundials/cvode.pxd' and ~/anaconda3/envs/skodes/lib/python3.7/site-packages/scikits/odes/sundials/cvode.pxd exist.

[bmcage]

@utsekaj42 Could not look at this today, for next week. Note that current master is not tested with python 3.7, see https://travis-ci.org/bmcage/odes/builds/576422107 I'm working on a new branch #106 where 3.7 would be added in the testing.

For your error, add a print to see what directory is searched in, the error is clearly

van_der_pol_fun.pyx:37:8: 'scikits/odes/sundials/cvode.pxd' not found

As long as the pxd is not found, it cannot work, scikits must be on the python path.

@cmashraf same for you, the pxd is not found, so it cannot work:

hello.pyx:5:0: 'scikits/odes/sundials/cvode/CV_RhsFunction.pxd' not found

[utsekaj42]

I added print(sys.path) to the file. Doing so doesn't show anything when running the cython command. However, I have included the path from simply running python and printing the path:

python -c "import sys; print(sys.path)"
['', '~/anaconda3/envs/skod/lib/python37.zip', '~/anaconda3/envs/skod/lib/
python3.7', '~/anaconda3/envs/skod/lib/python3.7/lib-dynload', '/home/jsturdy/.local/l
ib/python3.7/site-packages', '~/anaconda3/envs/skod/lib/python3.7/site-packages']

scikits.odes is installed at '~/anaconda3/envs/skod/lib/python3.7/site-packages/scikits/'

[bmcage]

You need cython -I ~/odes/ as that is the dir you have scikits in which you want to import.

So, you are testing the van der pol example. On my system, I obtain a bug in the sig, with current odes the function should be

cdef class CV_Rhs_van_der_pol_cy(CV_RhsFunction):
    cpdef int evaluate(self, DTYPE_t t,
                       np.ndarray[DTYPE_t, ndim=1] y,
                       np.ndarray[DTYPE_t, ndim=1] ydot,
                       object userdata = None) except? -1:

Note the extra except at the end! (bug from except @aragilar , to add, or will you remove except ...).

The sig error is after the import, so the import is working for me with an -I as I mention. I'm on python2.7 however, so to test further I need to set up python3 version on this PC. If my suggestion fixes it however, I don't need to do that.

[utsekaj42]

It seems both -I ~/odes and the except were needed. Thanks for the help.

The actual minimal set of includes I have been able use are as follows:

# $CONDA_PREFIX is ~/anaconda3/envs/skod where sundials, python, etc are all installed
# $CC points to the conda version of gcc

cython  -I ~/odes/ van_der_pol_fun.pyx # path to clone of odes that has not been installed with pip.

$CC -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing \
-I $CONDA_PREFIX/include/python3.7m \
-I $CONDA_PREFIX/include/ \
-I $CONDA_PREFIX/lib/python3.7/site-packages/numpy/core/include/ \
-o van_der_pol_fun.so van_der_pol_fun.c

and to run it save the following as test_van.py

from scikits.odes.odeint import odeint
import numpy as np
import van_der_pol_fun as vdp
y0 = [2, 0]
outtimes = np.linspace(0, 1000, 101)
sol2 = odeint(vdp.CV_Rhs_van_der_pol_cy(), outtimes, y0, method='cvode')
print(sol2)

and run python test_van.py.

I am a bit curious about the need for "-I ~/odes". I actually am not able to install from source in my environment (something to figure out at a later date), so odes is only installed from PyPI, so the scikit that is loaded to run in python is different from that used by cython to compile.

Naively, I would expect that it could be configured so that the necessary files for a cython RHS could be packaged in site-packages. Something to look into later. It seems to work for now.

[bmcage]

if all installed in site-packates, pointing -I to where odes is installed, so that import scikits works, should indeed be sufficient.

In your original post you do not have it. You have

cython  -I ~/anaconda3/envs/skodes/lib/python3.7/site-packages/scikits/odes/sundials/  \
        -I ~/odes/scikits/odes/sundials/ \
        -I ~/anaconda3/envs/skodes/include/cvode/ \
        van_der_pol_fun.pyx

You would need -I ~/anaconda3/envs/skodes/lib/python3.7/site-packages/ too, and the first two with odes explicit should not be needed.

[utsekaj42]

So should the following succeed?

cython  -I $CONDA_PREFIX/lib/python3.7/site-packages/ \
        -I $CONDA_PREFIX/include/cvode/ \
        van_der_pol_fun.pyx

It does not. I include the output of ls $CONDA_PREFIX/lib/python3.7/site-packages/scikits/odes/sundials and ls ~/odes/scikits/odes/sundials, as these are somewhat different and maybe the reason why -I ~/odes works.

ls /home/jsturdy/anaconda3/envs/jts/lib/python3.7/site-packages/scikits/odes/sundials/
c_cvode.pxd                                  c_sunmatrix.pxd
c_ida.pxd                                    cvode.cpython-37m-x86_64-linux-gnu.so
c_nvector_serial.pxd                         cvode.pxd
common_defs.cpython-37m-x86_64-linux-gnu.so  ida.cpython-37m-x86_64-linux-gnu.so
common_defs.pxd                              ida.pxd
c_sundials.pxd                               __init__.py
c_sunlinsol.pxd                              __pycache__

and

ls ~/odes/scikits/odes/sundials/
c_cvode.pxd           common_defs.c    c_sundials.pxd   cvode.c    ida.c    __init__.py
c_ida.pxd             common_defs.pxd  c_sunlinsol.pxd  cvode.pxd  ida.pxd  sundials_config.pxi
c_nvector_serial.pxd  common_defs.pyx  c_sunmatrix.pxd  cvode.pyx  ida.pyx