caffe
caffe copied to clipboard
BVLC
ImportError: dynamic module does not define module export function (PyInit__caffe)
[jalal@goku caffe]$ pwd
/scratch/caffe
$make clean
$mkdir build
$cd build
$cmake -DBLAS=open ..
$make install all -j 12
$sudo make install
(I get no error for above). The only change I have made in the code I got from git repo is copying Makefile.config.example to Makefile.config and changing a few things to reflect I am using opencv, Anaconda Python and also cudnn
[jalal@goku caffe]$ python
Python 3.5.4 |Anaconda custom (64-bit)| (default, Nov 9 2017, 00:39:12)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/scratch/caffe/python/caffe/__init__.py", line 1, in <module>
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "/scratch/caffe/python/caffe/pycaffe.py", line 13, in <module>
from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: dynamic module does not define module export function (PyInit__caffe)
>>> quit()
[jalal@goku caffe]$ tail -5 ~/.bashrc
#export PYTHONPATH=$HOME/caffe/python
export PYTHONPATH=/scratch/caffe/python
#export CAFFE_ROOT=$HOME/caffe
export PATH=/usr/local/anaconda3/bin/python:$PATH
I basically need to import caffe in the Anaconda Python 3.
ImportError: dynamic module does not define module export function (PyInit__caffe)
Here is the Makefile.config
[jalal@goku caffe]$ cat Makefile.config
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_60,code=sm_60 \
-gencode arch=compute_61,code=sm_61 \
-gencode arch=compute_61,code=compute_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/lib64/atlas #added by Mona
BLAS_LIB := /usr/lib64/atlas #added by Mona
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := /usr/local/anaconda3
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python3.5m \
$(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
[jalal@goku caffe]$
Please suggest what is wrong and how to fix it?
[jalal@goku caffe]$ uname -a Linux goku.bu.edu 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[jalal@goku caffe]$ lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.4.1708 (Core) Release: 7.4.1708 Codename: Core
>>> print(cv2.getBuildInformation())
General configuration for OpenCV 3.1.0 =====================================
Version control: unknown
Platform:
Host: Linux 4.8.0-46-generic x86_64
CMake: 3.6.3
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/gmake
Configuration: Release
C/C++:
Built as dynamic libs?: YES
C++ Compiler: /opt/rh/devtoolset-2/root/usr/bin/c++ (ver 4.8.2)
C++ flags (Release): -I/cs/software/anaconda3/include -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -I/cs/software/anaconda3/include -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG
C Compiler: /opt/rh/devtoolset-2/root/usr/bin/cc
C flags (Release): -I/cs/software/anaconda3/include -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fopenmp -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -I/cs/software/anaconda3/include -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release):
Linker flags (Debug):
Precompiled headers: YES
Extra dependencies: /cs/software/anaconda3/lib/libjpeg.so /cs/software/anaconda3/lib/libpng.so /cs/software/anaconda3/lib/libtiff.so /cs/software/anaconda3/lib/libhdf5.so /usr/lib64/librt.so /usr/lib64/libpthread.so /cs/software/anaconda3/lib/libz.so /usr/lib64/libdl.so /usr/lib64/libm.so dl m pthread rt
3rdparty dependencies: libwebp libjasper IlmImf libprotobuf
OpenCV modules:
To be built: core flann hdf imgproc ml photo reg surface_matching video dnn fuzzy imgcodecs shape videoio highgui objdetect plot superres xobjdetect xphoto bgsegm bioinspired dpm face features2d line_descriptor saliency text calib3d ccalib datasets rgbd stereo structured_light tracking videostab xfeatures2d ximgproc aruco optflow stitching python3
Disabled: world contrib_world
Disabled by dependency: -
Unavailable: cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java python2 ts viz cvv matlab sfm
GUI:
QT: NO
GTK+: NO
GThread : NO
GtkGlExt: NO
OpenGL support: NO
VTK support: NO
Media I/O:
ZLib: /cs/software/anaconda3/lib/libz.so (ver 1.2.8)
JPEG: /cs/software/anaconda3/lib/libjpeg.so (ver 80)
WEBP: build (ver 0.3.1)
PNG: /cs/software/anaconda3/lib/libpng.so (ver 1.6.27)
TIFF: /cs/software/anaconda3/lib/libtiff.so (ver 42 - 4.0.6)
JPEG 2000: build (ver 1.900.1)
OpenEXR: build (ver 1.7.1)
GDAL: NO
Video I/O:
DC1394 1.x: NO
DC1394 2.x: NO
FFMPEG: NO
codec: NO
format: NO
util: NO
swscale: NO
resample: NO
gentoo-style: NO
GStreamer: NO
OpenNI: NO
OpenNI PrimeSensor Modules: NO
OpenNI2: NO
PvAPI: NO
GigEVisionSDK: NO
UniCap: NO
UniCap ucil: NO
V4L/V4L2: YES/YES
XIMEA: NO
Xine: NO
gPhoto2: NO
Parallel framework: OpenMP
Other third-party libraries:
Use IPP: 9.0.1 [9.0.1]
at: /opt/conda/conda-bld/opencv_1491943704081/work/opencv-3.1.0/3rdparty/ippicv/unpack/ippicv_lnx
Use IPP Async: NO
Use VA: NO
Use Intel VA-API/OpenCL: NO
Use Eigen: YES (ver 3.2.8)
Use Cuda: NO
Use OpenCL: NO
Use custom HAL: NO
Python 2:
Interpreter: (ver 3.5.3)
Python 3:
Interpreter: /cs/software/anaconda3/bin/python (ver 3.5.3)
Libraries: /cs/software/anaconda3/lib/libpython3.5m.so (ver 3.5.3)
numpy: /cs/software/anaconda3/lib/python3.5/site-packages/numpy/core/include (ver 1.12.1)
packages path: /cs/software/anaconda3/lib/python3.5/site-packages
Python (for build):
Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Matlab: NO
Tests and samples:
Tests: NO
Performance tests: NO
C/C++ Examples: NO
Install path: /cs/software/anaconda3
cvconfig.h is in: /opt/conda/conda-bld/opencv_1491943704081/work/opencv-3.1.0/build
-----------------------------------------------------------------
>>> cv2.__version__
'3.1.0'
>>>
Hi monajalal, I also get the same error: import error: dynamic module does not define module export function (PyInit__caffe) Have you solved the error?
I have the same issue, but in my case it was working just after compiling source. I have exited a shell, and after opening it again problem started.
.bashrc
export PYTHONPATH=$HOME/caffe/python:$PYTHONPATH
Thank you so much. I'm now OK and my Caffe is working well.
On Fri, Feb 9, 2018 at 12:10 AM, sunblade [email protected] wrote:
I have the same issue, but in my case it was working just after compiling source. I have exited a shell, and after opening it again problem started.
.bashrc
export PYTHONPATH=$HOME/caffe/python:$PYTHONPATH
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/BVLC/caffe/issues/6054#issuecomment-364190105, or mute the thread https://github.com/notifications/unsubscribe-auth/AhLwKKpcBxVhprREenxeMY2Oxy_B7xDjks5tSzGjgaJpZM4QccHu .
@pencilzhang Did you fix it? I ran into the same dynamic error when import caffe.
I save my command for caffe installation. I have attached this file to you. I hope you'll OK.
On Thu, Mar 22, 2018 at 9:41 AM, WUhailing [email protected] wrote:
@pencilzhang https://github.com/pencilzhang Did you fix it? I ran into the same dynamic error when import caffe.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BVLC/caffe/issues/6054#issuecomment-375165144, or mute the thread https://github.com/notifications/unsubscribe-auth/AhLwKCupIg0lcauDPnrzpa1FFO1Y5HxGks5tgxZOgaJpZM4QccHu .
which conda conda info --envs source activate py36 sudo apt-get upgrade sudo apt-get install -y build-essential cmake git pkg-config sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev protobuf-compiler sudo apt-get install -y libatlas-base-dev sudo apt-get install -y --no-install-recommends libboost-all-dev sudo apt-get install -y libgflags-dev libgoogle-glog-dev liblmdb-dev sudo apt install build-essential conda install atlas boost gflags glog hdf5 leveldb lmdb openblas protobuf conda install tensorflow-gpu git clone https://github.com/BVLC/caffe cd caffe cp Makefile.config.example Makefile.config
Change the following in Makefile.config 1.Uncomment OpenCV3 OPENCV_VERSION := 3
- Comment the py2.7 path #PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
-
change for anaconda python3.6 ANACONDA_HOME := $(HOME)/anaconda3/envs/py36 PYTHON_INCLUDE := $(ANACONDA_HOME)/include
$(ANACONDA_HOME)/include/python3.6m
$(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include -
Change python_lib #PYTHON_LIB := /usr/lib PYTHON_LIB := $(ANACONDA_HOME)/lib ========================================================== make all make pycaffe make all mkdir build cd build cmake -D python_version=3 .. make all make install cd .. conda install cython scikit-image ipython h5py nose pandas protobuf pyyaml jupyter cd python sed -i -e 's/python-dateutil>=1.4,<2/python-dateutil>=2.0/g' requirements.txt cd caffe/python for req in $(cat requirements.txt); do pip install $req; done export PYTHONPATH=~/caffe/python/:$PYTHONPATH export PYTHONPATH=/home/dell/caffe/python:$PYTHONPATH echo $PYTHONPATH cd .. make runtest cd python import caffe
@nn-ww Thanks. It turns out to be 'cmake' issue, so switch to 'make', it works.
Thanks. I got 'import caffe' and it works well.
On Fri, Mar 23, 2018 at 4:37 PM, WUhailing [email protected] wrote:
@nn-ww https://github.com/nn-ww Thanks. It turns out to be 'cmake' issue, so switch to 'make', it works.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BVLC/caffe/issues/6054#issuecomment-375602576, or mute the thread https://github.com/notifications/unsubscribe-auth/AhLwKGnZdDM_oydneP6kJour25mSAy-Uks5thMlLgaJpZM4QccHu .
Thank you @nn-ww . I switch to cmake then everything works fine on a Mac OS system. @WUhailing
I've solved the problem. In my case, in the first anaconda envs, the protoc is not compatible with pycaffe (caffe), so I entered another envs.
This problem happens when you use python2 build with python3
Just uncomment:
# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python3 python3.5m
PYTHON_INCLUDE := /usr/include/python3.5m \
/usr/lib/python3.5/dist-packages/numpy/core/include
Hi @mrgloom , I got following error when i import cadquery package, this package are installed but i got error bellow,
import freecad import cadquery Traceback (most recent call last): File "/home/hp/.local/lib/python3.6/site-packages/cadquery/freecad_impl/init.py", line 151, in
import FreeCAD ModuleNotFoundError: No module named 'FreeCAD'
During handling of the above exception, another exception occurred:
import FreeCAD ImportError: dynamic module does not define module export function (PyInit_FreeCAD)
for futur readers, this error is caused because of the default python version is set to 3.x.x, I set it to python 2 using pyenv global system and then import caffe worked !
for what it's worth, I got this error because I had compiled Caffe for python2.7 but I'm running python 3. So I was in the inverse situation that @mjrlgue mentions above. The solution was to add -D python_version=3 flag to the cmake call.
git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . && \
pip install --upgrade pip && \
cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && \
mkdir build && cd build && \
cmake -DCPU_ONLY=1 -D python_version=3 .. && \
make -j2
(note that the above is for a CPU build)
@alex9311 This works for me, really appreciate that. My platform is Nvidia Jetson AGX Xavier.
