Distributions and Histograms

[omg777]

Hi, thanks for the nice work.

Can you explain about distributions and histograms in tensorboard ? How can I understand?

[bfarzin]

Have a look here and here to see more. I am not an expert with Tensorboard, but I have found it helpful to figure out what is going on in more complex models. The version checked in right now has this code disabled (it is commented out right now.) But you could remove the comments if you have TF installed and it should produce graphs of the loss and distributions of the matrix weights and gradients. I find those gradient histograms particularly helpful in that they tell you what kind of clipping might be helpful and if the gradients are converging as you proceed into training. The "out of the box" graphs seem to be good enough for my uses but, I believe, you can extended them even further if you want.

Let me know if there are other questions you have about the code or the TensorBoard use.