Nan in summary histogram for: D_X/fake

[victor-felicitas]

Hello,

I have a mistake after ~1900 steps:

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1226, in init self._traceback = _extract_stack() InvalidArgumentError (see above for traceback): Nan in summary histogram for: D_X/fake [[Node: D_X/fake = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](D_X/fake/tag, D_X_4/output/add/_1361)]]

Loading the checkpoint leads to same problem every time. I've read somewhere it could mean no convergence is observed. Here are details of the training:

INFO:root:-----------Step 1400:------------- INFO:root: G_loss : 4.85528945923 INFO:root: D_Y_loss : 0.0743376016617 INFO:root: F_loss : 4.97825098038 INFO:root: D_X_loss : 0.0959008038044

INFO:root:-----------Step 1900:------------- INFO:root: G_loss : 4.92536830902 INFO:root: D_Y_loss : 0.184448152781 INFO:root: F_loss : 4.78839635849 INFO:root: D_X_loss : 0.231521636248

INFO:root:-----------Step 2000:------------- INFO:root: G_loss : 5.98963975906 INFO:root: D_Y_loss : 0.09952506423 INFO:root: F_loss : 5.93069791794 INFO:root: D_X_loss : 0.215163201094

Any advice would be much appreciated!!!