Training Loss: NAN (after some training) Test Loss: NAN (always)

[MuhammadAsadJaved]

[MuhammadAsadJaved]

Hi, I am training YOLOV3 with 1 class and dataset is about 7000 images.

The training loss decrease for few iterations and then stop decreasing, test loss is always NAN.
I am using default settings and anchors here are my configurations and loss reports. What is the possible reasons and how to overcome this problem?

[MrZhousf]

change np.mean to np.nanmean like this: train_epoch_loss, test_epoch_loss = np.nanmean(train_epoch_loss), np.nanmean(test_epoch_loss)

[MuhammadAsadJaved]

@MrZhousf OK. Thank you so much. I'll give it a try.