keras-cv icon indicating copy to clipboard operation
keras-cv copied to clipboard

COCO metrics only evaluated on image_ids with predictions

Open digital-nomad-cheng opened this issue 1 year ago • 1 comments

https://github.com/keras-team/keras-cv/blob/94b0a551d03ee9de3d81663e2c1a680cb113f7f1/keras_cv/src/metrics/coco/pycoco_wrapper.py#L223-L227

The pycoco_wrapper only evaluates on images where there is a prediction, and leave out image_ids where there is not prediction. This is a serious bug if I understand it correctly. The coco metrics should be evaluated on all the images. Correct way to do this should be below, without filtering image_ids.

    coco_dt = coco_gt.loadRes(predictions=coco_predictions)
    coco_eval = COCOeval(coco_gt, coco_dt, iouType="bbox")

digital-nomad-cheng avatar Nov 08 '24 15:11 digital-nomad-cheng

Thanks for reporting the issue! We have consolidated the development of KerasCV into the new KerasHub package, which supports image, text, and multi-modal models. Please read the announcement. KerasHub will support all the core functionality of KerasCV.

KerasHub can be installed with !pip install -U keras-hub. Documentation and guides are available at keras.io/keras_hub.

With our focus shifted to KerasHub, we are not planning any further development or releases in KerasCV. If you encounter a KerasCV feature that is missing from KerasHub, or would like to propose an addition to the library, please file an issue with KerasHub.

sachinprasadhs avatar May 07 '25 22:05 sachinprasadhs