BERT-NER icon indicating copy to clipboard operation
BERT-NER copied to clipboard

WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 25 vs previous value: 25. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize

Open agarwalishan opened this issue 3 years ago • 0 comments

I am doing NER using BERT for past months on google colab GPU and everything was working fine but now when I am doing same on CPUs I am getting this warning. When am using colab GPU for training then there is no issue and no warnings of this kind. But when I am training with same data and same parameters I am getting this type of warning on running on colab CPU and on my local system's CPU and training never finishes.

WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 25 vs previous value: 25. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.

Can someone please tell me the reason why problem occurs only on CPU not on GPU keeping the configurations same? And how can I resolve it? By the way BERT is using AdamOptimizer and I have never modified BERT's optimizer.py .

agarwalishan avatar Aug 04 '20 09:08 agarwalishan