TensorFlow2.0-Examples icon indicating copy to clipboard operation
TensorFlow2.0-Examples copied to clipboard

YOLOv3 model doesn't converge

Open JVD9kh96 opened this issue 4 years ago • 7 comments

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

JVD9kh96 avatar Jul 28 '21 00:07 JVD9kh96

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

I met the same situation,Have you solved?

zimengxueying avatar Sep 01 '21 05:09 zimengxueying

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

I met the same situation,Have you solved?

I solved the problem by using tensorflow 2.2.0. It turned out that in tensorflow 2.5.0 which I was using, after the first prediction, the weights somehow got random and the only true prediction was for the first test image and then the weights got random. When I used tensorflow 2.2.0 the issue was solved. however, I was able to solve the issue in tensorflow 2.5.0 by freezing the weghts after loading the model.

I was gonna close the issue, so let me know if your problem is solved.

JVD9kh96 avatar Sep 23 '21 08:09 JVD9kh96

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

I met the same situation,Have you solved?

I solved the problem by using tensorflow 2.2.0. It turned out that in tensorflow 2.5.0 which I was using, after the first prediction, the weights somehow got random and the only true prediction was for the first test image and then the weights got random. When I used tensorflow 2.2.0 the issue was solved. however, I was able to solve the issue in tensorflow 2.5.0 by freezing the weghts after loading the model.

I was gonna close the issue, so let me know if your problem is solved.

When I run video_demo.py, I met the same problem. Only the first prediction is true, the others are nan. I guess the problem is in the 'utils.load_weights(model, "./yolov3.weights")'. How to solve the problem? I use tensorflow==2.7.0

mastergao57 avatar Mar 13 '22 14:03 mastergao57

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

I met the same situation,Have you solved?

I solved the problem by using tensorflow 2.2.0. It turned out that in tensorflow 2.5.0 which I was using, after the first prediction, the weights somehow got random and the only true prediction was for the first test image and then the weights got random. When I used tensorflow 2.2.0 the issue was solved. however, I was able to solve the issue in tensorflow 2.5.0 by freezing the weghts after loading the model.

I was gonna close the issue, so let me know if your problem is solved.

What command did you added and where? Thanks!

giakoumoglou avatar Jun 06 '22 18:06 giakoumoglou

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

I met the same situation,Have you solved?

I solved the problem by using tensorflow 2.2.0. It turned out that in tensorflow 2.5.0 which I was using, after the first prediction, the weights somehow got random and the only true prediction was for the first test image and then the weights got random. When I used tensorflow 2.2.0 the issue was solved. however, I was able to solve the issue in tensorflow 2.5.0 by freezing the weghts after loading the model. I was gonna close the issue, so let me know if your problem is solved.

When I run video_demo.py, I met the same problem. Only the first prediction is true, the others are nan. I guess the problem is in the 'utils.load_weights(model, "./yolov3.weights")'. How to solve the problem? I use tensorflow==2.7.0

One way that helped me fix this issue was down-grading tensorflow to 2.2.0. Another solution wich I found was freezing the weights after loading them. These two solutions was helpful for me.

JVD9kh96 avatar Jun 06 '22 18:06 JVD9kh96

Can you paste the command that freezes the weights?

giakoumoglou avatar Jun 06 '22 18:06 giakoumoglou

Hello. First of all, thank you for your amazing code. While trying to train the YOLOv3 model from scratch on yymnist dataset, I noticed that the loss is fluctuating between 20 and 40 and it doesn't converge. I didnt't change any of the config settings except the _C.YOLO.classes = "data/classes/voc.names" which I changed it to _C.YOLO.classes = "data/classes/yymnist.names". Am I doing something wrong? I use tensorflow==2.5.0

I met the same situation,Have you solved?

I solved the problem by using tensorflow 2.2.0. It turned out that in tensorflow 2.5.0 which I was using, after the first prediction, the weights somehow got random and the only true prediction was for the first test image and then the weights got random. When I used tensorflow 2.2.0 the issue was solved. however, I was able to solve the issue in tensorflow 2.5.0 by freezing the weghts after loading the model. I was gonna close the issue, so let me know if your problem is solved.

What command did you added and where? Thanks!

After you load the weights with model.set_weights(weights) or model.load_weights("path/to/weights"), just add the following command: model.trainable = False

JVD9kh96 avatar Jun 06 '22 18:06 JVD9kh96