benchmarks
benchmarks copied to clipboard
Branch for each TensorFlow release?
Will there not be a cnn_tf_vX.Y_compatible branch for each release of TensorFlow? If there is not a 'compatible' branch for a new TF release, does that mean the release will work fully with the most recent 'compatible' branch?
The 1-to-1 mapping of the last few releases was very convenient and clear. Benchmarking TF 1.11 by using the cnn_tf_v1.10_compatible branch is a little unintuitive. Particularly when you are writing instructions for an inexperienced TensorFlow user to confirm their install by using the benchmarks, it's nice for those instructions to apply to future TF versions without requiring the user to understand anything about the benchmark repo.
Is creating a 'compatible' branch for each TensorFlow release an option - even if it's just essentially a pointer to a previous branch? From where I'm sitting, it looks like it would be a nice little convenience without any cost.
I would recommend inexperienced users to not use tf_cnn_benchmarks as it shows neat ways to use tensorflow to get performance but it can be hard to follow. For your request what we need to do is ask our DevOps team to tag our repo like they do tensorflow/models. The tag would be done for each release of TensorFlow, we stay aligned.
I will take this issue it may take some time to bubble up the request and update their "run book".
Agreed we should set this up with DevOps to automatically create them. In the mean time, I will manually update them. Sorry, I have forgotten to create 1.11 and 1.12 branches. I will create them now, and if future TF releases are released, like 1.13, without a cnn_tf_vX.Y_compatible branch, please file a new issue.