'tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/libcc_ops.lo', needed by 'main/dede'

[abhiguru]

find reveals libc_ops.lo in local-py3-opt

/home/zzzz/ws/deepdetect/build/tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local-py3-opt/bin/tensorflow/cc/libcc_ops.lo

I was thinking of changing the CMakeList.txt TF_LIB_DEPS to use local-py3-opt is that a viable solution or was the build done incorrect I followed the instructions on the setup page

[beniz]

Hi, can you specify which platform you are building on, what deepdetect commit, your version of bazel and whether you are building TF for GPUs.

[beniz]

My guess is that you are building on Fedora or CentOS, and in that case you may try to modify the CMakeLists.txt...

[abhiguru]

I did change the CMakeList and it worked okay, I had tough time with curlpp and tcmalloc_minimal but it built okay the second time tcmalloc_minimal.so.4 has to be symlinked to tcmalloc_minimal.so on Ubuntu 16.10 just in case anyone comes looking. (Building with TF and CPU Only)

[beniz]

OK thanks, good to know! curlpp is still built with the wrong GCC version in 14.04 and 16.04... Same in 16.10 from what you are saying...

[abhiguru]

apt-get installed libcurlpp-dev gave linking errors for some reason I had to build from source and install followed by a fresh build

[beniz]

Yes sounds like same issue as #126 regarding curlpp.

[fedegarza]

Hi, I have the same error no rule to make target 'tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/libcc_ops.lo', needed by 'main/dede'

but I can't find any of libcc_ops.lo, nor libconst_op.a, nor libops.a, nor libscope.a

Can you help me? Thanks!

[fedegarza]

Sorry I'm building on Ubuntu 14.04, Deepdetect master and Bazel 0.4.4 Thanks!

[beniz]

Try with bazel 0.4.3, it's a TF thing.

[fedegarza]

Thanks! it works!

[rdoume]

Hi, It appears I have the same issue here, but my bazel version is the 0.4.3 I'm on 16.04, TF without the GPU. I'm on deepdetect master with commit : e99ee48f94678214b0a84a063059437056558032

Do you have any clues ?

[beniz]

@rdoume share your full cmake call, etc... I haven't noticed any difficulties on our builds, but I'll look again. Make sure you don't have multiple bazel installs or something.

[rdoume]

Hi @beniz, I run the following cmake commande : cmake .. -DUSE_XGBOOST=0N -DUSE_TF=ON -DUSE_TF_CPU_ONLY=ON

The first issue I encountered was that some TF files (such as image_ops.h) were left in the bazel-genfile and not the tensorflow folder. After pasting them in the tf folder, the compilation works until that linking issue. What else should I provide as information? Thanks

[beniz]

@rdoume I've tested with bazel 5.0 on Ubuntu 16.04 and it fails as reported in this ticket. However, downgrading to bazel 0.4.3 fixes the issue as reported. So double-check your current version of bazel, and check that you don't have one already running that would be of a later version. Again, I believe this is a TF configuration thing.

[rdoume]

@beniz I still encounter this issue. the result of the "bazel version" command gives me this :

Build label: 0.4.3
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 22 12:31:25 2016 (1482409885)
Build timestamp: 1482409885
Build timestamp as int: 1482409885

My installation process here is :

git clone https://github.com/beniz/deepdetect
cd deepdetect
mkdir build
cd build 
cmake .. -DUSE_XGBOOST=0N -DUSE_TF=ON -DUSE_TF_CPU_ONLY=ON

which give me this

-- The C compiler identification is GNU 5.4.0 -- The CXX compiler identification is GNU 5.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- Boost version: 1.58.0 -- Found the following Boost libraries: -- filesystem -- thread -- system -- iostreams -- chrono -- date_time -- atomic -- regex -- CUDA detected: 8.0 -- Added CUDA NVCC flags for: sm_61 -- Fetching Tensorflow -- OpenCV 2 (2.4.9.1) found (/usr/share/OpenCV) -- Configuring customized caffe -- Configuring XGBoost -- Build Tests : OFF -- Configuring done -- Generating done -- Build files have been written to: /home/robin/deepdetect/build

Then I make. While tensorflow is building I have this :

[ 6%] No patch step for 'tensorflow' [ 8%] No update step for 'tensorflow' [ 10%] Performing configure step for 'tensorflow' Note: checking out 'v1.0.1'.

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at e895d5c... Fixing 2 minor issues as suggested by Gunan. checking file tensorflow/BUILD Unreversed patch detected! Ignore -R? [n] Apply anyway? [n]

I say no to both, then I give the TF instalation the location of the file needed ( like where is python etc.)

Then I got this error:

/home/deepdetect/src/tfinputconns.h:30:41: fatal error: tensorflow/cc/ops/image_ops.h: No such file or directory

As I see, it's because the files generated in bazel-genfile folder are not pasted in the tensorflow folder. If I do this manualy and restart the make, the making continue

Then I get the error of this issue :

make[2]: *** No rule to make target 'tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/libcc_ops.lo', needed by 'main/dede'. Stop.

But what I notice is that the local_linux-opt is pointing to a folder (local-opt) that does not exist. If I manualy build the folder, then paste the files from the bazel bin inside that local-op/bin/tensorflow, I can continue the make.

But then the linking fail :

[ 16%] Built target tensorflow [ 32%] Built target xgboost [ 48%] Built target caffe_dd [ 95%] Built target ddetect [ 97%] Linking CXX executable dede /usr/bin/ld: warning: libcublas.so.8.0, needed by /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libcusparse.so.8.0, needed by /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libcurand.so.8.0, needed by /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so, not found (try using -rpath or -rpath-link) /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDdot_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cusparseCreateMatDescr' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDestroy_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDasum_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandSetGeneratorOffset' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cusparseCreate' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDgemm_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasSasum_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasCreate_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandGenerateUniform' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandCreateGenerator' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasSdot_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDcopy_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandGenerateNormalDouble' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasSgemm_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDgemv_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasSscal_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandGenerateNormal' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cusparseDestroyMatDescr' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cusparseDestroy' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cusparseSetMatIndexBase' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDscal_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasSaxpy_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cusparseSetMatType' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandGenerateUniformDouble' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasDaxpy_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasScopy_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to cublasSgemv_v2' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandSetPseudoRandomGeneratorSeed' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to curandDestroyGenerator' /home/robin/deepdetect/build/caffe_dd/src/caffe_dd/build/lib/libcaffe.so: undefined reference to `curandGenerate' collect2: error: ld returned 1 exit status main/CMakeFiles/dede.dir/build.make:131: recipe for target 'main/dede' failed

I'm a bit puzzled by all this.

[beniz]

You are just need to fix you LD path, look the main README up.

[rdoume]

Huh, okay that last mistake is on me, my LD_LIBRARY_PATH was defined as /usr/local/cuda-8.0/ and not cuda, as my python tensorflow is using this one.

Ok so, It build now. But it's a bit a pain to do all these copy-pasting bazel folders in order to properly build. Thank you for your time though !

[beniz]

@rdoume you shouldn't need to move file around, the build is automated but for the few TF Y/n questions. If you can lay out the exact thing you do (e.g. moving files around), put it down here, as it may help others, including ourselves if we see this issue somewhere. Thanks.

[rdoume]

Okay. So to solve this, and as stated in my previous comment : I start like in the readme :

git clone https://github.com/beniz/deepdetect
cd deepdetect
mkdir build
cd build 
cmake .. -DUSE_XGBOOST=0N -DUSE_TF=ON -DUSE_TF_CPU_ONLY=ON

After answering the Tensorflow configuration, the build starts until this error:

~/deepdetect/src/tfinputconns.h:30:41: fatal error: tensorflow/cc/ops/image_ops.h: No such file or directory

To fix this one, I merge the content of the folder deepdetect/build/tensorflow/src/tensorflow/bazel-genfiles/tensorflow/ with this one : ~/deepdetect/build/tensorflow/src/tensorflow/tensorflow/

Then restart the building with the make commande This carry on the build, until this error:

make[2]: *** No rule to make target 'tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local_linux-opt/bin/tensorflow/cc/libcc_ops.lo', needed by 'main/dede'. Stop.

This error was the puzzling one. Looking at the ~/deepdetect/build/tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/ folder I could see that the local_linux_opt was pointing towards a folder that was inexistant. So I had to do

mkdir ~/deepdetect/build/tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local-opt
mkdir ~/deepdetect/build/tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local-opt/bin

then I paste then this folder (the folder itself) ~/deepdetect/build/tensorflow/src/tensorflow/bazel-bin/tensorflow inside the newly created ~/deepdetect/build/tensorflow/src/tensorflow/dbuild/execroot/tensorflow/bazel-out/local-opt/bin

Then I carry on the building with the make command, and now the build is complete. (After checking the $LD_LIBRARY_PATH contains /usr/local/cuda/lib64 and not /usr/local/cuda-8.0/lib64)