correlation.so : Undefined symbols (after succesfull compilation)

[PetreanuAndi]

Hello,

My setup is python 3.5.2 , TF 1.8.0 and Cuda 9.0. I have successfully compiled flownet in order to use the correlation layer. However, when loading the correlation.so, I get the following error :

from object_detection.utils.correlation import correlation ImportError: /data/tensorflow/models/research/object_detection/utils/correlation.so: undefined symbol: _ZTIN10tensorflow8OpKernelE

After looking this up online, i found more than a couple of people suggesting this fix : https://www.tensorflow.org/versions/master/extend/adding_an_op#compile_the_op_using_your_system_compiler_tensorflow_binary_installation

( adding -ltensorflow_framework to the CGPU Flags)

I have successfully compiled (make all) flownet with the added flag, but i still get the same error. This is the output of my compilation process (only included last several lines, but no errors occur, as you can see, and the flag is in place) :

g++ -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0 "src/ops/downsample/downsample_kernel.cc" "src/ops/downsample/downsample_op.cc" src/ops/build/downsample_kernel_gpu.o -pthread -shared -fPIC -L/usr/local/cuda/lib -L/usr/local/cuda/lib64 -lcudart -L/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow -ltensorflow_framework -o src/ops/build/downsample.so /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from .conv import register_converters as register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. nvcc -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI -D_GLIBCXX_USE_CXX11_ABI=0 -c src/ops/correlation/correlation_kernel.cu.cc -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -o src/ops/build/correlation_kernel_gpu.o /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/generated_message_reflection.h(685): warning: variable "unused" was set but never used

nvcc -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0 -c src/ops/correlation/correlation_grad_kernel.cu.cc -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -o src/ops/build/correlation_grad_kernel_gpu.o /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/generated_message_reflection.h(685): warning: variable "unused" was set but never used

nvcc -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0 -c src/ops/correlation/pad.cu.cc -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -o src/ops/build/correlation_pad_gpu.o /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/generated_message_reflection.h(685): warning: variable "unused" was set but never used

g++ -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0 "src/ops/correlation/correlation_kernel.cc" "src/ops/correlation/correlation_grad_kernel.cc" "src/ops/correlation/correlation_op.cc" src/ops/build/correlation_kernel_gpu.o src/ops/build/correlation_grad_kernel_gpu.o src/ops/build/correlation_pad_gpu.o -pthread -shared -fPIC -L/usr/local/cuda/lib -L/usr/local/cuda/lib64 -lcudart -L/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow -ltensorflow_framework -o src/ops/build/correlation.so # /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from .conv import register_converters as register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. nvcc -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI -D_GLIBCXX_USE_CXX11_ABI=0 -c "src/ops/flow_warp/flow_warp.cu.cc" -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -o "src/ops/build/flow_warp_gpu.o" /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(57): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(304): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(305): warning: integer conversion resulted in a change of sign

/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow/include/google/protobuf/generated_message_reflection.h(685): warning: variable "unused" was set but never used

nvcc -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0 -c "src/ops/flow_warp/flow_warp_grad.cu.cc" -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -o "src/ops/build/flow_warp_grad_gpu.o" /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications. src/ops/flow_warp/flow_warp_grad.cu.cc(59): warning: variable "ch_off" was declared but never referenced

src/ops/flow_warp/flow_warp_grad.cu.cc(59): warning: variable "ch_off" was declared but never referenced

g++ -g -std=c++11 -Ipython -c "import tensorflow; print(tensorflow.sysconfig.get_include())" -I"/usr/local/cuda/include" -DGOOGLE_CUDA=1 -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0 "src/ops/flow_warp/flow_warp_op.cc" "src/ops/flow_warp/flow_warp.cc" "src/ops/flow_warp/flow_warp_grad.cc" "src/ops/build/flow_warp_gpu.o" "src/ops/build/flow_warp_grad_gpu.o" -pthread -shared -fPIC -L/usr/local/cuda/lib -L/usr/local/cuda/lib64 -lcudart -L/home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/tensorflow -ltensorflow_framework -o "src/ops/build/flow_warp.so" /home/andi/miniconda3/envs/tf18/lib/python3.5/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications.

---

Please help :"> This is very important to me, as I am trying hard to implement a recent paper with great results. Changing the TF and CUDA versions would probably be a solution, but i-m only considering that as my last option.

Guys, fingers crossed :+1:

[rabitdash]

Maybe you can change this line in MakeFile

CFLAGS += -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=0

to

CFLAGS += -D_MWAITXINTRIN_H_INCLUDED -D_FORCE_INLINES -D__STRICT_ANSI__ -D_GLIBCXX_USE_CXX11_ABI=1

this does help for me.

[pringithub]

Ever figure this out? I'm having the same problem and @rabitdash 's solution does not work for me :/

[chenrxi]

it does not work for me either. sad