YOLOv6 icon indicating copy to clipboard operation
YOLOv6 copied to clipboard

Why is the IOU_LOSS as nan?

Open joel5638 opened this issue 2 years ago • 5 comments

Im training with 69 images and 10 images for testing. Im trainng on my local pc.

im using python3 tools/train.py --batch 1 --conf configs/yolov6s.py --data data/dataset.yaml --device 0

Im getting iou_loss as nan as shown in the picture im sharing. Screenshot from 2022-07-13 13-17-41

joel5638 avatar Jul 13 '22 07:07 joel5638

Hi, the possible reason is that the gt box of some samples is too large and the value overflows in the calculation of ciou loss. You can try to solve this problem by adding “loss = torch.nan_to_num(loss)” to the “IOUloss” function.

qinhao14 avatar Jul 13 '22 08:07 qinhao14

@qinhao14 where do i add that? right in the function?

joel5638 avatar Jul 13 '22 08:07 joel5638

@joel5638 YOLOv6/yolov6/utils/figure_iou.py class IOUloss: image

qinhao14 avatar Jul 13 '22 09:07 qinhao14

while training. i dont see the 69 images being loaded. I see this. Is this normal. Screenshot from 2022-07-13 15-38-10

joel5638 avatar Jul 13 '22 10:07 joel5638

Yes, in you screenshot, it shows Final number of valid images: 69 / labels:69.

mtjhl avatar Jul 14 '22 02:07 mtjhl