Polygon coordinates for the mask of each detected object

[iresusharma]

Hi i was wondering that can we get the coordinates of the polygon mask generated of the each detected object using Mask_RCNN.? if so how can we get that.?

[hydeta]

https://github.com/matterport/Mask_RCNN/blob/3deaec5d902d16e1daf56b62d5971d428dc920bc/mrcnn/model.py#L2482

using the detect method should return the masks as one of the keys in the result dictionary

[iresusharma]

yes i have tried it, its returning boolean values for masks.. and when i converted it into as int.. its returning 0,0,0... what i want is like we have coordinates for bounding boxes (something like this [ 95 74 402 341]) same if i can get for the generated polygons!

[konstantin-frolov]

You have 2 ways:

1. Use skimage find_contours or cv2.findContours for each big mask in results[0]['masks'][i].
2. You can crop area from results[0]['masks'][i] by coordinates in results[0]['rois'][i]. Thenuse skimage find_contours or cv2.FindContours and shift contour coordinates on results[0]['rois'][i][0:2]

[sohinimallick]

Hi i was wondering that can we get the coordinates of the polygon mask generated of the each detected object using Mask_RCNN.? if so how can we get that.?

Hey did you manage to do this??

[sohinimallick]

yes i have tried it, its returning boolean values for masks.. and when i converted it into as int.. its returning 0,0,0... what i want is like we have coordinates for bounding boxes (something like this [ 95 74 402 341]) same if i can get for the generated polygons!

The predicted rois are giving such values but I don't know if that is what is required

[pasquale90]

To obtain the contour (perimeter) of the binary mask, you could try something like this:

def mask2polygon(image,masks):
    _,_,numasks=masks.shape
    image_copy = image.copy()
    polygons=[]
    if numasks==0:
        print('Null predictions')
        return []
    elif numasks!=0:
        print(f'Number of masks predicted : {numasks}')
        for i in range(numasks):
            mask_reformed=masks[:,:,i].astype("uint8")
            if np.sum(mask_reformed==1): # if mask is not empty - i.e. filled with zeros only, then proceed ..  
                contours, _ = cv2.findContours(image=mask_reformed,mode=cv2.RETR_EXTERNAL, method=cv2.CHAIN_APPROX_SIMPLE) # or you can also method=cv2.CHAIN_APPROX_NONE
                
                # check the contours
                cv2.drawContours(image=image_copy, contours=contours, contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA) # check the contours

                polygons.append(contours)

     # check the contours
    cv2.imwrite(savepath,image_copy) 
    return polygons

# main function
image=cv2.imread(impath)
results = model.detect([image], verbose=1)
polygons=mask2polygon(image,results[0]['masks']) # image is passsed as an param just for checking the results of the function - you may remove it latter