deepdetect
deepdetect copied to clipboard
jolibrain
Build error on Fedora
I'm getting the following error when I try to build... I installed all boost related rpms to no avail. I tried googling everywhere and I don't see much related to server.hpp
In file included from <path>/deepdetect/src/deepdetect.cc:25:0:
<path>l/deepdetect/src/httpjsonapi.h:26:50: fatal error: boost/network/protocol/http/server.hpp: No such file or directory
#include <boost/network/protocol/http/server.hpp>
^
compilation terminated.
src/CMakeFiles/ddetect.dir/build.make:62: recipe for target 'src/CMakeFiles/ddetect.dir/deepdetect.cc.o' failed
make[2]: *** [src/CMakeFiles/ddetect.dir/deepdetect.cc.o] Error 1
CMakeFiles/Makefile2:122: recipe for target 'src/CMakeFiles/ddetect.dir/all' failed
make[1]: *** [src/CMakeFiles/ddetect.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Hmm, this is used by cppnetlib. First I'd recommend to compile cppnetlib by hand as packaged versions are most often pretty old. Look ate the Ubuntu 15.10 and 16.04 deepdetect build issues in case this could be useful. Another handle is to look where the server.hpp file lies on your system. I'm not familiar with Fedora but boost is often broken into small subpackages and you just miss one here. Let me know how this goes, and if you can share your steps here it would be useful to other. Also start without GPU / Cuda, it'd be easier for a first build.
Note: to get up and running quickly, I'd suggest the docker builds.
Thanks for the quick response!
I had originally tried the docker image but was getting errors when trying to train a text classifier.. I'm not very familiar with docker so I thought it'd be easier to debug if I ran the server locally.
I'm trying with docker again and attaching to the container
I'm running this to train:
python dd_train.py --model-repo ./sbren --training-repo ./sbren --sname sbren --tsplit 0.1 --base-lr 0.01 --sequence 1014 --iterations 150000 --test-interval 1000 --nclasses 38
and I can see the following error in the container
E1104 19:10:11.987526 20 caffemodel.cc:69] error reading or listing caffe models in repository ./sbren
and I've tried various combinations for the model-repo and training-repo and none of them have worked, I'm thinking because the image can't see my filesystem?
Also, to verify... the model-repo will be the output directory and the training-repo is the "input" directory, correct?
I think the server tells you what the problem is: the server is looking locally within the container for your model repo but my guess is that it is fact on your host. Look at the DD +docker doc, it tells you how to share a directory with the container. Let me know if it is not clear.
ahh great thanks! I got it training now.
I'll go back to trying to get the build to work though, I'll keep this open until I figure out the build issue
Well, I compiled cpp-netlib manually and copied it, and asio, into src. This got me past my previous error, but now I'm getting:
[ 33%] Built target caffe_dd
[ 37%] Building CXX object src/CMakeFiles/ddetect.dir/deepdetect.cc.o
In file included from /home/rdu/difernan/Devel/deepdetect/src/services.h:29:0,
from /home/rdu/difernan/Devel/deepdetect/src/apistrategy.h:26,
from /home/rdu/difernan/Devel/deepdetect/src/deepdetect.h:25,
from /home/rdu/difernan/Devel/deepdetect/src/deepdetect.cc:22:
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h: In member function ‘void dd::ImgInputFileConn::transform(const dd::APIData&)’:
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:320:4: error: ‘mt19937’ is not a member of ‘std’
std::mt19937 g;
^~~
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:322:6: error: ‘g’ was not declared in this scope
g = std::mt19937(_seed);
^
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:322:10: error: ‘mt19937’ is not a member of ‘std’
g = std::mt19937(_seed);
^~~
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:325:8: error: ‘random_device’ is not a member of ‘std’
std::random_device rd;
^~~
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:326:8: error: ‘g’ was not declared in this scope
g = std::mt19937(rd());
^
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:326:12: error: ‘mt19937’ is not a member of ‘std’
g = std::mt19937(rd());
^~~
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:326:28: error: ‘rd’ was not declared in this scope
g = std::mt19937(rd());
^
/home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:328:47: error: ‘g’ was not declared in this scope
std::shuffle(_images.begin(),_images.end(),g); //XXX beware: labels are not shuffled, i.e. let's not shuffle while testing
^
In file included from /home/rdu/difernan/Devel/deepdetect/src/caffeinputconns.h:28:0,
from /home/rdu/difernan/Devel/deepdetect/src/imginputfileconn.h:370,
from /home/rdu/difernan/Devel/deepdetect/src/services.h:29,
from /home/rdu/difernan/Devel/deepdetect/src/apistrategy.h:26,
from /home/rdu/difernan/Devel/deepdetect/src/deepdetect.h:25,
from /home/rdu/difernan/Devel/deepdetect/src/deepdetect.cc:22:
looking at some stuff on SO I think I need -std=c++0x, but I tried CXXFLAGS="-std=c++0x" make and that didn't work.
gcc --version
gcc (GCC) 6.2.1 20160916 (Red Hat 6.2.1-2)
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
OK, two things:
- try adding
#include <random>to the headers ofimginputfileconn.h - if that doesn't work, do you have any way to test with gcc5.
I haven't tested gcc6 yet, but I'll do it soon.
#include <random> in imginputfileconn.h got my past that point! also had to add it to txtinputfileconn.h but now I'm getting:
[ 50%] Building CXX object src/CMakeFiles/ddetect.dir/txtinputfileconn.cc.o
/home/rdu/difernan/Devel/deepdetect/src/txtinputfileconn.cc: In member function ‘void dd::TxtInputFileConn::parse_content(const string&, const float&)’:
/home/rdu/difernan/Devel/deepdetect/src/txtinputfileconn.cc:266:13: error: ‘utf8’ has not been declared
c = utf8::next(str_i,end);
^~~~
/home/rdu/difernan/Devel/deepdetect/src/txtinputfileconn.cc: In member function ‘void dd::TxtInputFileConn::build_alphabet()’:
/home/rdu/difernan/Devel/deepdetect/src/txtinputfileconn.cc:350:15: error: ‘utf8’ has not been declared
uint32_t c = utf8::next(str_i,end);
I had originally had an error about utf8.h missing so I had manually downloaded this into src:
wget https://raw.githubusercontent.com/sheredom/utf8.h/master/utf8.h
ahhh duh, I needed utf8cpp-devel... google failed me :)
Now I'm getting:
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:138:37: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
void fillup_response(http_server::response &response,
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:221:18: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
http_server::response &response)
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc: In member function ‘void APIHandler::fillup_response(int&, const JDoc&, std::__cxx11::string&, int&, const string&)’:
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:209:29: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
response = http_server::response::stock_reply(http_server::response::status_type(outcode),stranswer);
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:209:64: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
response = http_server::response::stock_reply(http_server::response::status_type(outcode),stranswer);
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:210:14: error: request for member ‘headers’ in ‘response’, which is of non-class type ‘int’
response.headers[1].value = "application/json";
^~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:213:11: error: request for member ‘headers’ in ‘response’, which is of non-class type ‘int’
response.headers.resize(3);
^~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:214:11: error: request for member ‘headers’ in ‘response’, which is of non-class type ‘int’
response.headers[2].name = "Content-Encoding";
^~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:215:11: error: request for member ‘headers’ in ‘response’, which is of non-class type ‘int’
response.headers[2].value = "gzip";
^~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:217:14: error: request for member ‘status’ in ‘response’, which is of non-class type ‘int’
response.status = static_cast<http_server::response::status_type>(code);
^~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:217:48: error: ‘response’ in ‘http_server {aka struct boost::network::http::server<APIHandler>}’ does not name a type
response.status = static_cast<http_server::response::status_type>(code);
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:217:56: error: expected ‘>’ before ‘::’ token
response.status = static_cast<http_server::response::status_type>(code);
^~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:217:56: error: expected ‘(’ before ‘::’ token
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:217:56: error: ‘::status_type’ has not been declared
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:217:76: error: expected ‘)’ before ‘;’ token
response.status = static_cast<http_server::response::status_type>(code);
^
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc: In member function ‘void APIHandler::operator()(const request&, int&)’:
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:250:26: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
response = http_server::response::stock_reply(http_server::response::not_found,_hja->jrender(_hja->dd_not_found_404()));
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:250:61: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
response = http_server::response::stock_reply(http_server::response::not_found,_hja->jrender(_hja->dd_not_found_404()));
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:350:26: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
response = http_server::response::stock_reply(http_server::response::not_found,_hja->jrender(_hja->dd_not_found_404()));
^~~~~~~~
/home/rdu/difernan/Devel/deepdetect/src/httpjsonapi.cc:350:61: error: ‘boost::network::http::http_server<APIHandler>::response’ has not been declared
response = http_server::response::stock_reply(http_server::response::not_found,_hja->jrender(_hja->dd_not_found_404()));
^~~~~~~~
Looks like https://github.com/performous/performous/issues/201.
Could you try against 0.11.2 at the moment ?
Due to #207 users are testing the forthcoming 0.13 and they successfully built againt it on Ubuntu 16.04. I'll try 0.12 + gcc6.
I tried 0.11.2 and I think it got me past that (I did a make clean at some point), but now I'm getting:
/usr/bin/ld: cannot find -lcppnetlib-uri
collect2: error: ld returned 1 exit status
main/CMakeFiles/dede.dir/build.make:121: recipe for target 'main/dede' failed
make[2]: *** [main/dede] Error 1
CMakeFiles/Makefile2:177: recipe for target 'main/CMakeFiles/dede.dir/all' failed
make[1]: *** [main/CMakeFiles/dede.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
btw, what I've been doing is symlinking the boost dir from the cpp-netlib dir into the src dir so it builds it all together.
have you tried locating this lib in the cppnetlib directory ? It could be found in `build/libs/network/src. One you have it, adding its path to LD_LIBRARY_PATH should do the trick.
All I see in there is are static libs:
ls ~/cpp-netlib/build/libs/network/src
CMakeFiles cmake_install.cmake CTestTestfile.cmake libcppnetlib-client-connections.a libcppnetlib-server-parsers.a libcppnetlib-uri.a Makefile
I tried with LD_LIBRARY_PATH pointing there but nothing
hmm, one way could be to directly add the lib to https://github.com/beniz/deepdetect/blob/master/main/CMakeLists.txt. Look how its done for static libs in tests/cmakelists.txt
WOOOOO THAT WORKED!!! You are amazing, thanks for all the prompt feedback!
I changed the last line to
target_link_libraries (dede ddetect ${CUDA_LIB_DEPS} glog gflags ${OpenCV_LIBS} curlpp curl crypto ssl ${Boost_LIBRARIES} ${CAFFE_LIB_DEPS} ${XGBOOST_LIB_DEPS} ${TF_LIB_DEPS} $HOME/cpp-netlib/build/libs/network/src/libcppnetlib-uri.a)
I'll try to write up how I built it on Fedora 24 in case it's helpful for you or others
So I just re-built it with tests enabled...
When I run make the first time, I get:
/usr/bin/ld: cannot find -lcaffe
collect2: error: ld returned 1 exit status
main/CMakeFiles/dede.dir/build.make:122: recipe for target 'main/dede' failed
make[2]: *** [main/dede] Error 1
CMakeFiles/Makefile2:177: recipe for target 'main/CMakeFiles/dede.dir/all' failed
make[1]: *** [main/CMakeFiles/dede.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
but when I run make again, it finishes fine... thinking it might be an issue with order of LD deps?
However, when I run ctests I get:
Test project /home/aiguofer/deepdetect/build/tests
No tests were found!!!
Maybe it is the make -j that fails compiling the Caffe proto file before the C++ files. This can happen depending on your hardware configuration.
Check that you have something in build/tests.
Ahh, yeah running without -j4 makes the compilation run fine.
And here's what's in build/tests:
ls tests
CMakeFiles Testing cmake_install.cmake CTestTestfile.cmake Makefile
Looks like the tests are not activated. Most likely this is due to missing gtest package.