Validation: single device = core

[larissa95]

NOTE: To reiterate- in this context, "single device" means using a single CPU core or single GPU, not a single machine. Similarly, "multi-device" does not refer to multiple machines, but to multiple CPU cores and/or GPUs. See "3.3 Distributed Execution" for multiple machine discussion.

Are you sure about that? In the config I can specify as many devices as I want to, by default TensorFlow only finds one CPU device.

[samjabrahams]

Not sure which part of the quote you're asking clarification for. Which part do you think needs to be adjusted? I think it's incorrect to say "CPU core" instead of just a single CPU chip, but I want to make sure I understand the issue you're raising.

[larissa95]

I would be extremely helpful to have a clear definition what a device is. Is a device a single processing unit (no "real" concurrency possible)? You can define as many devices as you want by doing the following:

config = tf.ConfigProto(device_count={"CPU": 2},
                        inter_op_parallelism_threads=2,
                        intra_op_parallelism_threads=1)
sess = tf.Session(config=config)

How is it possible that you can define as many as you want despite having only one processor with 4 cores?

[samjabrahams]

That's a good question: I can't say I know with certainty off the top of my head. From reading the comments in config.proto, it looks like device_count doesn't tell TensorFlow how many devices there are, but rather what the maximum amount of each device it should use. That is, if you had two GPUs, but only wanted to use one, you could set device_count={"GPU": 1} to force TensorFlow to only use one.

I think this might be a good thing to post on the TensorFlow mailing list to see if you can get a more definitive answer.