how to build single library file and some issues

[dshawul]

I am building the tf r1.11 library using bazel on windows. It compiles fine but the libtensorflow_cc.so.if.lib file I get is much smaller (in kb) than yours that is in MB. Also I am not able to link properly against a simple program without getting "unresolved errors". I think that is because the single lib file I have does not have everything. How do you get one lib file (tensorflow.lib) that has everything needed in tensorflow.dll(or libtensorflow_cc.so) ?

Also I see that you have a tensorflow_static.lib file even for the GPU verion. How do you produce that ?

Btw the only reason I am trying to recompile (instead of using your binaries) is because tensorflow built with --config=monolithic seems to have problems with protobuf strings like https://github.com/tensorflow/tensorflow/issues/5379 https://github.com/tensorflow/tensorflow/issues/16291 . The fix is to add ptotobuf; in tf_version_script linker file. I want to rebuild with that since it is a very serious issue that doesn't allow you to do many things without. Try this simple code given in the second link that demonstrates all string fields share the same memory location if you use shared library built with --config=monolithic (tried it boths on linus and windows).

#include "tensorflow/core/protobuf/control_flow.pb.h"
#include "tensorflow/core/protobuf/config.pb.h"
#include <iostream>

int main() {
  tensorflow::CondContextDef options;
  tensorflow::GPUOptions gpu_options;

  gpu_options.set_allocator_type("allocator_type");
  options.set_context_name("context");
  options.set_pred_name("pred");
  options.set_pivot_name("pivot");
  options.set_branch(1);

  std::cout << "context name " << options.context_name() << std::endl; //print pivot
  std::cout << "pred name " << options.pred_name()<< std::endl; //print pivot
  std::cout << "pivot name " << options.pivot_name()<< std::endl; //print pivot

  std::cout << "allocator_type " << gpu_options.allocator_type() << std::endl; //print pivot

}

[fo40225]

follow https://www.tensorflow.org/install/install_sources_windows to prepare the environment.

git clone https://github.com/tensorflow/tensorflow.git -b v1.11.0-rc1
cd tensorflow
python ./configure.py
...
bazel build --config=opt --config=cuda //tensorflow:libtensorflow_cc.so

You will get the libtensorflow_cc.so in tensorflow\bazel-out\x64_windows-opt\bin\tensorflow.

libtensorflow_cc.so is a dll, gen the .lib from it and link it and look if it works or not.

[dshawul]

Thank you for your quick response. Note that I have already built the library that way. My problem is that the .lib file it produces is much smaller than your produced by cmake. And I can not properly link a simple program with this smaller lib while I have no problems with yours. The dll (so) files are about the same size though. Did you do something in your cmake build to have a single tensorflow.lib (not tensorflow.dll) that you would need to link against any program

[dshawul]

I think in your cmake you have CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS which is probably why it is much larger than mine and also good for linking. If I do a dumpbin /exports libtensorflow_cc.so on the bazel created so, I get much less exported symbols than dumpbin /exports tensorflow.dll on your dll.

[fo40225]

Maybe use --config=monolithic and https://github.com/tensorflow/tensorflow/blob/v1.10.0/tensorflow/contrib/cmake/tools/create_def_file.py to gen shared lib from static lib is a way to work.

[ttdd11]

@fo40225 With this process can you assign a specific GPU on windows using tensorflow_cc.dll?

[fo40225]

@ttdd11 https://github.com/tensorflow/tensorflow/commit/0c8deb2f9178feef562320688198885b23ffd647 The instruction in commit message may work, I haven't try it.

[ttdd11]

@fo40225 These look they they are for tensorflow.dll not tensorflow_cc.dll. The issue I'm having is that I can't assign a graph to a specific gpu. Have you been able to achieve this?

[ttdd11]

@fo40225 But generally to make the tensorflow_cc.dll and .lib you are using the bazel build instructions. How are you exporting the correct symbols?

[fo40225]

@ttdd11 Using the bazel build instructions above only generate the .dll with no .lib file and .dll contains only C symbol not C++.

The commit not only fix the libtensorflow.so (C lib) but also fix the libtensorflow_cc.so (C++ lib).

Just try to checkout the commit and bazel build --config=opt //tensorflow:tensorflow_cc.

good luck

[ttdd11]

@fo40225 Have you been able to set device in the c_api? I think this have been broken for about a year.

[fo40225]

@ttdd11 try to use the file in https://github.com/fo40225/tensorflow-windows-wheel/tree/master/1.10.0/cpp to see if it helps or not

or just wait google totally fix building cpp lib of tensorflow on windows https://github.com/tensorflow/tensorflow/pull/26152

[ttdd11]

@fo40225 I need to build with CUDA 10 for rtx support. How do I test the branch in the second link? doesn't look like its in yet.