main_loop_tf
main_loop_tf copied to clipboard
Fix wiggle lines in TB when experiment is resumed
The model checkpoints and tensorboard events are not saved with the same frequency. When the model is reloaded and training resumes this causes the main loop to write new events to the event file that have the same global step (x-coordinate) as some previous event. As a result the TB graph looks weird. This can be fixed by means of using SessionLog messages as suggested in the documentation.
I don't have time to work on this right now, but I'd be happy to review a PR.