TensorFlow.jl icon indicating copy to clipboard operation
TensorFlow.jl copied to clipboard

`allow_growth` doesn't seem to work

Open Evizero opened this issue 8 years ago • 5 comments

The python documentation mentions an option to prevent tensorflow of allocating all the GPU memory at once (https://www.tensorflow.org/tutorials/using_gpu#allowing_gpu_memory_growth).

The equivalent julia code doesn't seem have any effect. The following still allocates the whole GPU according to nvidia-smi

config = TensorFlow.tensorflow.ConfigProto()
config.gpu_options = TensorFlow.tensorflow.GPUOptions()
config.gpu_options.allow_growth = true
session = Session(Graph(), config)

Any idea what is going on? Could it be that I am just misunderstanding the intend of allow_growth ?

Evizero avatar Sep 18 '17 08:09 Evizero

You are right about the intended usage, and it should work. I'll check it out.

malmaud avatar Sep 21 '17 18:09 malmaud

Because Session(allow_growth=true) works, here is how this method does it:

config = TensorFlow.tensorflow.ConfigProto()
gpu_options = TensorFlow.tensorflow.GPUOptions()
ProtoBuf.set_field!(gpu_options, :allow_growth, true)
ProtoBuf.set_field!(config, :gpu_options, gpu_options)
Session(Graph(), config)

The big difference being the ProtoBug.set_field!

Evizero avatar Sep 30 '17 10:09 Evizero

Oh yes, that's a limitation of ProtoBuf.jl and ultimately the lack of dot-overloading in Julia. So now I remember why I wrote the convenience allow_growth keyword option in the first place.

malmaud avatar Sep 30 '17 15:09 malmaud

Personally I think this workaround is "good enough". Feel free to close this issue.

Evizero avatar Sep 30 '17 15:09 Evizero

Let's document it somewhere before closing.

malmaud avatar Oct 02 '17 20:10 malmaud