yolact icon indicating copy to clipboard operation
yolact copied to clipboard

Cannot eval on COCO test-dev on Google Colab

Open Maxinho96 opened this issue 5 years ago • 9 comments

I am trying to use YOLACT++ from Google Colab. Inference on images/videos works fine, but I can't evaluate on COCO test-dev. I run this command: python eval.py --trained_model=weights/yolact_plus_resnet50_54_800000.pth --dataset=coco2017_testdev_dataset And I get:

Traceback (most recent call last):
 File "eval.py", line 1105, in <module>
  evaluate(net, dataset)
 File "eval.py", line 936, in evaluate
  img, gt, gt_masks, h, w, num_crowd = dataset.pull_item(image_idx)
 File "/content/yolact/data/coco.py", line 172, in pull_item
  if target.shape[0] == 0:
AttributeError: 'NoneType' object has no attribute 'shape'

If I change line 172 in coco.py to check if target is None I get another error:

Traceback (most recent call last):
  File "eval.py", line 1105, in <module>
    evaluate(net, dataset)
  File "eval.py", line 956, in evaluate
    prep_metrics(ap_data, preds, img, gt, gt_masks, h, w, num_crowd, dataset.ids[image_idx], detections)
  File "eval.py", line 390, in prep_metrics
    gt_boxes = torch.Tensor(gt[:, :4])
TypeError: 'NoneType' object is not subscriptable

I can get past this error by adding the argument --output_coco_json, but I don't think it's the right way to use the script. I think it's a bug but could also be a Colab related issue (or some error of mine).

Maxinho96 avatar Mar 11 '20 14:03 Maxinho96

@Maxinho96 I commented out ''if target.shape[0] == 0: ...''

Serge-weihao avatar Mar 11 '20 15:03 Serge-weihao

@Maxinho96 I commented out ''if target.shape[0] == 0: ...''

Did it work? I get the second error.

Maxinho96 avatar Mar 11 '20 16:03 Maxinho96

this solves the first problem and generate two json file, but i meet another problem(those files are too large for my model output)

Serge-weihao avatar Mar 11 '20 16:03 Serge-weihao

the mask_detections.json is 947M, it is impossible to upload it to the test server @dbolya

Serge-weihao avatar Mar 11 '20 16:03 Serge-weihao

I have just realised that the second error is normal because regular evaluation is not supported with COCO test-dev because it has no ground truth, so you can only evaluate it with json file + test server. BTW the first error still remains...

Maxinho96 avatar Mar 11 '20 21:03 Maxinho96

You can use --score_threshold=0.05 to reduce the file size. As for the first bug, that's an oversight on my part. For now you can comment it out. I'll try to get a fix out today.

dbolya avatar Mar 12 '20 13:03 dbolya

i got the same error, how to solves?

zhepherd avatar Mar 19 '20 10:03 zhepherd

i got the same error, how to solves?

Comment these lines: https://github.com/dbolya/yolact/blob/092554ad707c2749631dfe545c8a953b2b3f4a68/data/coco.py#L172-L174

and use --output_coco_json flag.

Maxinho96 avatar Mar 19 '20 12:03 Maxinho96

i got the same error, how to solves?

Comment these lines: https://github.com/dbolya/yolact/blob/092554ad707c2749631dfe545c8a953b2b3f4a68/data/coco.py#L172-L174

and use --output_coco_json flag.

For me, I add another condition based on the previous one as below: "if target.shape[0] == 0 and self.has_gt:"

ShihuaHuang95 avatar Aug 11 '20 09:08 ShihuaHuang95