amazon-dsstne icon indicating copy to clipboard operation
amazon-dsstne copied to clipboard

Support for Tesla V100, running SM 7.0

Open rajserc opened this issue 7 years ago • 12 comments

Getting Error: invalid device function launching kernel kScaleAndBias_kernel

Looks like only following is supported

enum SM_VERSION { SM_3X, SM_5X, SM_6X, };

What can be done to workaround this issue ?

rajserc avatar Dec 16 '17 01:12 rajserc

You can fix this in about 2 minutes by adding

"-gencode arch=compute_70,code=sm_70 "

to lines 13 and 17 of src/amazon/dsstne/Makefile.inc

We already ported to CUDA 9, the code is Volta-compatible, but CUDA 9 was prerelease when we did so and this would break on CUDA 8.

On Dec 15, 2017 5:40 PM, "rajserc" [email protected] wrote:

Getting Error: invalid device function launching kernel kScaleAndBias_kernel

Looks like only following is supported

enum SM_VERSION { SM_3X, SM_5X, SM_6X, };

What can be done to workaround this issue ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/amzn/amazon-dsstne/issues/148, or mute the thread https://github.com/notifications/unsubscribe-auth/ARNK9mvMc-btfTiQh1FVWO8Em4Y7hZ25ks5tAx-VgaJpZM4RELql .

scottlegrand avatar Dec 16 '17 01:12 scottlegrand

Thanks. BTW, i tried to build the dockerfile and jsoncpp dependancy is not in the dockerfile that causes the docker build fail.

Tried adding following, # JsonCpp setup RUN cd /tmp && \ wget https://github.com/open-source-parsers/jsoncpp/archive/svn-import.tar.gz && \ tar xvfz svn-import.tar.gz && \ cd jsoncpp-svn-import && \ mkdir -p build/release && \ cd build/release && \ cmake -DCMAKE_BUILD_TYPE=release -DJSONCPP_LIB_BUILD_SHARED=OFF -G "Unix Makefiles" ../.. && \ make -j 8 && \ make install && rm -rf /tmp/*

getting In file included from NNTypes.cpp:14:0: NNTypes.h:25:31: fatal error: jsoncpp/json/json.h: No such file or directory #include <jsoncpp/json/json.h>

The file is in /usr/local/include/json/json.h

What should i change to build the image ?

rajserc avatar Dec 16 '17 03:12 rajserc

#include <json/json.h>

ought to do the trick...

On Fri, Dec 15, 2017 at 7:16 PM, rajserc [email protected] wrote:

Thanks. BTW, i tried to build the dockerfile and jsoncpp dependancy is not in the dockerfile that causes the docker build fail.

Tried adding following,

JsonCpp setup RUN cd /tmp && \ wget https://github.com/open-

source-parsers/jsoncpp/archive/svn-import.tar.gz && \ tar xvfz svn-import.tar.gz && \ cd jsoncpp-svn-import && \ mkdir -p build/release && \ cd build/release && \ cmake -DCMAKE_BUILD_TYPE=release -DJSONCPP_LIB_BUILD_SHARED=OFF -G "Unix Makefiles" ../.. && \ make -j 8 && \ make install && rm -rf /tmp/*

getting In file included from NNTypes.cpp:14:0: NNTypes.h:25:31: fatal error: jsoncpp/json/json.h: No such file or directory #include <jsoncpp/json/json.h>

The file is in /usr/local/include/json/json.h

What should i change to build the image ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/amzn/amazon-dsstne/issues/148#issuecomment-352156878, or mute the thread https://github.com/notifications/unsubscribe-auth/ARNK9pUoaO4QEiJsIpB56_Es7r8PzcOhks5tAzX9gaJpZM4RELql .

scottlegrand avatar Dec 16 '17 03:12 scottlegrand

Got past that, but is there version dependency for jsoncpp or is it a linker issue not sure

getting NNNetwork.cpp: In function 'NNNetwork* LoadNeuralNetworkJSON(const string&, uint32_t, const std::vector<NNDataSetBase*>&)': NNNetwork.cpp:2763:59: error: 'class Json::ValueIterator' has no member named 'name' string name = itr.name(); ^ make[1]: Leaving directory `/opt/amazon/dsstne/src/amazon/dsstne/engine' make[1]: *** [NNNetwork.o] Error 1 make: *** [lib/libdsstne.a] Error 2

rajserc avatar Dec 16 '17 05:12 rajserc

Are you on Ubuntu 16.04 or 14.04?

On Dec 15, 2017 9:24 PM, "rajserc" [email protected] wrote:

Got past that, but is there version dependency for jsoncpp or is it a linker issue not sure

getting NNNetwork.cpp: In function 'NNNetwork* LoadNeuralNetworkJSON(const string&, uint32_t, const std::vector<NNDataSetBase*>&)': NNNetwork.cpp:2763:59: error: 'class Json::ValueIterator' has no member named 'name' string name = itr.name(); ^ make[1]: Leaving directory `/opt/amazon/dsstne/src/amazon/dsstne/engine' make[1]: *** [NNNetwork.o] Error 1 make: *** [lib/libdsstne.a] Error 2

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/amzn/amazon-dsstne/issues/148#issuecomment-352162401, or mute the thread https://github.com/notifications/unsubscribe-auth/ARNK9p_eMU7TUbxx7_OlLIAcywOkm_yjks5tA1QWgaJpZM4RELql .

scottlegrand avatar Dec 16 '17 06:12 scottlegrand

According to dockerfile it's Ubuntu 14

rajserc avatar Dec 16 '17 13:12 rajserc

having the same problem. If you managed to fix it please help by sharing! Thx

ashley915 avatar Jan 10 '18 08:01 ashley915

unfortunately not yet

rajserc avatar Jan 17 '18 03:01 rajserc

So the challenge here is that the version of JSONCPP in Ubuntu is really old and the function "name" used to be called "memberName". Sadly, I don't know any workaround for this since there's no way to detect the version of JSONCPP at compile-time. We are about 1 week away from updating the dockerfile, but you could try basing it on 8.0-cudnn7-devel-ubuntu16.04 instead and replacing all the downloads and installs with the appropriate apt-gets which are I believe: libopenmpi-dev, libnetcdf-dev, libnetcdf-c++4-dev, libjsoncpp-dev, libopenblas-dev, and libcppunit-dev.

scottlegrand avatar Jan 17 '18 05:01 scottlegrand

I'm not using docker. Will it still work without Docker?

ashley915 avatar Jan 22 '18 02:01 ashley915

With docker you need to be in ubuntu 16.04 as there was some backward incompatible changes made with JsonCPP

rgeorgej avatar Jan 22 '18 17:01 rgeorgej

Did this get resolved? We have long since checked in Volta support for DSSTNE and it works on a Vanilla Ubuntu 16.04 installation.

scottlegrand avatar Jun 14 '18 15:06 scottlegrand