tf-coriander icon indicating copy to clipboard operation
tf-coriander copied to clipboard

How can i install on windows

Open ShuvenduRoy opened this issue 7 years ago • 38 comments

I have Radoen Rx 470 gpu install on windows 10. Do i have any option for running tensorflow on this gpu

ShuvenduRoy avatar May 31 '17 17:05 ShuvenduRoy

Windows ... I'm not sure... I havnet tried. There are a few parts:

  • CLBlast => works on Windows, almost certainly
  • clang/llvm => seems to be available for Windows: http://releases.llvm.org/download.html#4.0.0
  • pthreads => pthreads is available for Windows . eg http://www.sourceware.org/pthreads-win32/

I think everything else should be fine. Do you want to try, see how far you can get, report any issues that arise? You'll probably need to fix the issues yourself, but I might be able to give some pointers, possibly.

Edit:

Sticky progress update, using msvc 2015, on branch winbuild of coriander, doc at https://github.com/hughperkins/coriander/blob/winbuild/doc/installation_windows_inprogress.md:

  • [X] installed an msvc 2015 box
  • [X] clew builds
  • [X] clblast builds
  • [X] easycl builds
  • [X] llvm headers build/built
  • [X] migrate coriander to c++11 threads
  • [X] can link against llvm
  • [X] can build cocl library
  • [X] can build ir-to-opencl tool
  • [X] can build patch-hostside tool
  • [X] clang++ working, can compile and run basic c++11 programs
  • [X] migrate cocl script etc, from bash to python
  • [ ] can run coriander test scripts
  • [ ] tf-coriander pip_install_package builds
  • [ ] tf-coriander wheel builds
  • [ ] tf-coriander wheel imports ok
  • [ ] tf-coriander simple tensorflow script works
  • [ ] tf-coriander test results match those on Ubuntu 16.04 and Mac Sierra

hughperkins avatar May 31 '17 22:05 hughperkins

oh wait, this is the tensorflow repository. Hmmm... well... I think the first thing would be to try to get coriander working first, ie https://github.com/hughperkins/coriander . Once we have coriander working, we can think about tensorflow itself. How does that sound?

hughperkins avatar May 31 '17 23:05 hughperkins

(seems like tensorflow itself supports windows , https://www.tensorflow.org/install/install_windows , so as long as coriander works ok on windows, we should stand a chance that tf-coriander works too)

hughperkins avatar May 31 '17 23:05 hughperkins

I don't really know how to build tf-coriander on windows either...

Ray-Eldath avatar Jun 08 '17 05:06 Ray-Eldath

Step 1: download clew, https://github.com/hughperkins/clew check that does/doesn't build, report any issues here.

On 8 June 2017 06:49:19 BST, Eldath [email protected] wrote:

I don't really know how to build tf-coriander on windows either...

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/hughperkins/tf-coriander/issues/38#issuecomment-307005340

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

hughperkins avatar Jun 08 '17 06:06 hughperkins

Yeah, I build clew successfully. clewTest returned num platforms: 1 that looks like a normal return. What should I do next ?

Ray-Eldath avatar Jun 08 '17 08:06 Ray-Eldath

Cool :-) . Next would be to try the next level up:

https://github.com/hughperkins/EasyCL

(EasyCL uses clew, under the hood)

hughperkins avatar Jun 08 '17 08:06 hughperkins

Still build successfully, and gpuinfo return correct devices info. Next ?

Ray-Eldath avatar Jun 08 '17 08:06 Ray-Eldath

cool :-) . Next one will be harder :P :

https://github.com/hughperkins/coriander

No-one has built this on Windows yet, as far as I know. You are entering uncharted territory :-)

hughperkins avatar Jun 08 '17 08:06 hughperkins

Finally, it generated cocl and cocl_wrapped file in dist and when I execute it in Cygwin it return:

Eldath@DESKTOP-RB6STNM ~ $ /cygdrive/f/Code/Research/coriander-master/dist/bin/cocl_wrapped
cocl args:
Usage: /cygdrive/f/Code/Research/coriander-master/dist/bin/cocl_wrapped [-c] [-g] [[-Iincludepath]...] [-o outfile] [infile.cu]

I don't know this means build successfully or not ;-)

Ray-Eldath avatar Jun 08 '17 10:06 Ray-Eldath

You're saying, the cmake and build bit finished successfully? (I know cocl wont run yet, but I'm quite pleasantly surprised if the cmake and build bit finished.

How are you building by the way? cygwin? mingsys? visual studio? something else?

hughperkins avatar Jun 08 '17 10:06 hughperkins

oh, cygdrive :-)

Hmmmm

Well, then there's a good chance many things will work as-is. You can test by doing things like:

git pull
make tests
make run-cuda_sample

edit: or simply:

../bin/cocl ../test/cocl/cuda_sample.cu
./cuda_sample

hughperkins avatar Jun 08 '17 12:06 hughperkins

The cmake works fine. It returns

Configuring done
Generating done

And when I build the project with cmake -DBUILD_TYPE=Release -P cmake_install.cmake (after solve a lot of problems) then it noticed me

CMake Error at cmake_install.cmake:202 (file):
  file INSTALL cannot find
  "F:/Code/Research/coriander-master/build/Release/cocl.dll".

Ray-Eldath avatar Jun 08 '17 12:06 Ray-Eldath

All right I'll test it first.

Ray-Eldath avatar Jun 08 '17 12:06 Ray-Eldath

cocl is a linux bash file so I only can run this in WSL... but console return an error:

./cocl: line 373: /usr/lib/llvm-3.6/bin/clang++: No such file or directory

How to fix ? And wether run in WSL is equals to run in Windows ?

Ray-Eldath avatar Jun 08 '17 13:06 Ray-Eldath

  • you need to use llvm-4.0. You need to download this manually from http://llvm.org/
  • if you are using the make installd /usr/local/bin/cocl, this is just a wrapper that sets CLANG_HOME, then calls /usr/local/bin/cocl_wrapped (or similar)
  • if you are using ../bin/cocl directly, you need to define CLANG_HOME=wherever/llvm-4.0/is
  • if you are running tests with cmake, you can define CLANG_HOME inside the ccmake curses ui

In your case, you probably need to set CLANG_HOME to point to wherever llvm-4.0 directory is, and you probably need to download this first.

hughperkins avatar Jun 08 '17 13:06 hughperkins

(Just in case it's useful, I created a Dockerfile, https://github.com/hughperkins/tf-coriander/blob/master/docker/Dockerfile It wont be useful for running of course, since wont be able to use opencl, but does show a full download, build, for ubuntu 16.04. might be helpful-ish? )

hughperkins avatar Jun 08 '17 16:06 hughperkins

I run command in Windows PowerShell manual but when I execute

bazel build ${LOCALRESOURCES} ${BUILDTYPE}  @grpc//:grpc_cpp_plugin
bazel build ${LOCALRESOURCES} ${BUILDTYPE} @protobuf//:protoc

it returns:

ERROR: no such package '@grpc//': The repository could not be resolved.
INFO: Elapsed time: 0.739s

and:

ERROR: no such package '@protobuf//': The repository could not be resolved.
INFO: Elapsed time: 0.850s

How to solve this? ;-|

Ray-Eldath avatar Jun 09 '17 06:06 Ray-Eldath

Thats a bazel issue. How sure are you coriander is working first? I mean, fixing coriander won't fix this bug. But I'd rather go bit by bit. Can you run all the tests included with coriander, and provide the results please?

On 9 June 2017 07:47:11 BST, Eldath [email protected] wrote:

I run command in Windows PowerShell manual but when I execute

bazel build ${LOCALRESOURCES} ${BUILDTYPE}  @grpc//:grpc_cpp_plugin
bazel build ${LOCALRESOURCES} ${BUILDTYPE} @protobuf//:protoc

it returns:

ERROR: no such package '@grpc//': The repository could not be resolved.
INFO: Elapsed time: 0.739s

and:

ERROR: no such package '@protobuf//': The repository could not be
resolved.
INFO: Elapsed time: 0.850s

How to solve this? ;-|

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/hughperkins/tf-coriander/issues/38#issuecomment-307309862

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

hughperkins avatar Jun 09 '17 06:06 hughperkins

Hi @hughperkins Does https://github.com/hughperkins/clew must need VS2010 on windows? Is there any lighter substituter ?

eromoe avatar Jun 15 '17 07:06 eromoe

@eromoe Should build on anything I would think. Might need -std=c++, but some base c++ might be sufficient actually. [edit: actually, c99 is probably sufficient, dont think it uses c++ at all actually]

That's probably true for EasyCL too. [edit: well, easycl uses c++, probably c++98 is sufficient though, no need for -std=c++11

Coriander needs -std=c++11. So msvc2010 is not sufficinet. You probably need msvc2015, if you're giong to go msvc route. Otherwise, you could use cygwin or msys, if you prefer, plausibly.

What compiler do you normally prefer to use? Or, do you mean, you'd prefer to have pre-built Windows binaries?

hughperkins avatar Jun 15 '17 07:06 hughperkins

Actually, I am not familiar with c++. I know cygwin ad msys are something linux emulators. But they looks lighter than msvc2015 , so I may prefer to. But I wonder , are the building steps as same as ubuntu if I choose cygwin or msys? I just hope it is not so complex.

PS: The building enviroment usually make me headache. There are many different settings or dependencies on each platfom/env . I always admire your guys who can write the right config and steps . What I know is just follow the installation guide, once got error directly game over and cry for help. Do you have any suggesttion for my situation?

eromoe avatar Jun 15 '17 07:06 eromoe

Looks like there are commandline tools available, dont need full ide, http://landinghub.visualstudio.com/visual-cpp-build-tools

hughperkins avatar Jun 15 '17 13:06 hughperkins

Note: Coriadne build on Windows is in progress on branch 'winbuild', of Coriander, https://github.com/hughperkins/coriander/tree/winbuild

hughperkins avatar Jun 16 '17 11:06 hughperkins

(and doc is here: https://github.com/hughperkins/coriander/blob/winbuild/doc/installation_windows_inprogress.md )

hughperkins avatar Jun 16 '17 13:06 hughperkins

Update:

  • setup msvc2015 build box
  • easycl, clew, clblast build ok
  • llvm headers build/built ok
  • working on migrating coriander to c++11 threads, since they're the future, and saves adding pthreads to the windwos build
  • status sticky here: https://github.com/hughperkins/tf-coriander/issues/38#issuecomment-305339404

hughperkins avatar Jun 17 '17 11:06 hughperkins

Hmmm, as far as which compiler to use when building application source-code...

  • clang++ seems to best support mingw headers and runtime
  • but tensorflow windows build seems to be using msvc2015, as far as I can tell

hughperkins avatar Jun 18 '17 07:06 hughperkins

Seems like the clang++, from http://releases.llvm.org/download.html#4.0.0 can at least build cuda_sample.cu into NVIDIA® CUDA™ bytecode, using msvc 2015 headers:

screen shot 2017-06-18 at 8 43 45 am screen shot 2017-06-18 at 8 44 05 am

Warnings are ok, compile errors would be less good.

Coriander binaries are building ok now, on Windows 64-bit, using msvc2015.

Next step is to modify the bash script at https://github.com/hughperkins/coriander/blob/master/bin/cocl , to be a python script.

hughperkins avatar Jun 18 '17 07:06 hughperkins

Update: on the winbuild branch, there is a new command available cocl_py, instead of cocl, which uses python, instead of bash, so should be portable, cross-platform. The endtoend coriander tests pass using this. The eigen tests fail for some reason. I'm not going to look into why just now, but some simple comparison of the options being fed to clang++ in one with those fed to clang++ by the other should be sufficient to fix that. I think I'm going to take a braek from the Windows thing for a while, and look at eg thrust instead, so please feel free to pick up the reins on this for a bit :-)

(Edit: by the way, this commit reverts from using cocl_py as the default, back to cocl, so I can merge to master-ish. To set it to default, once it's fully working on eigne tests, just do the reverse of what is in this commit basically)

hughperkins avatar Jun 18 '17 17:06 hughperkins

(finally I have merged winbuild branch onto master, with the cocl script now marked as deprecated, and cocl_py the preferred entrypoint)

hughperkins avatar Jun 18 '17 18:06 hughperkins