Where do you apply DenseCRF to the score map?

[UdonDa]

Hello， thank you for sharing your nice project.

In sec3.4， you write about how to get the instance masks， but I do not know where you apply the DenseCRF...

For example,

we first generate the score map for the detected box using the trained segmentation branch.  
Then, the predicted score map of the box is pasted to a map Sˆ of the same size as the input image according to the box’s location.  

is corresponding to the first score map and paste pate

And，values predicted by the model are returned in this part

All the above codes are run in this inference part.

Finally， the encoded masks are outputed in the encode part.

However， I do not know the DenseCRF part...

For setting up DenseCRF, we employ the map Sˆ as the unary term and use the color and pixel location differences with the bilateral kernel to construct the pairwise term.
After optimization using mean field approximation, DenseCRF produces the final instance mask

Please teach us， thank you.

[chengchunhsu]

Hi UdonDa,

Thanks for asking.

But for now, the post-processing (DenseCRF) part has not been included in the repository. The DenseCRF code is based on Matlab and was developed by another author. Unfortunately, I currently have no access to his server.

I will upload them next week (after I return from the NeurIPS conference).

Cheng-Chun

[UdonDa]

Thanks for replying.

You implement the CRF part by MATLAB, ok. So, run_test function in [here] (https://github.com/chengchunhsu/WSIS_BBTP/blob/master/tools/train_net.py#L79)do not work now, right?

I think that It's easier to implement CRF by pydencecrf.

After you upload the part, I will try!

Please do your best NeurlPS. Thanks.

[chengchunhsu]

Hi UdonDa,

Actually, the run_test function will evaluate the performance of the trained model "without" post-processing (DenseCRF).

If you have implemented the post-processing part in python. You can let me know by opening an issue or commits. Thanks!

Cheng-Chun

[theodoruszq]

Hi @UdonDa , what is your mask AP on VOC 2012 validation set.

We strictly follow the author's settings, however, we cannot reproduce the results.

Box AP:

AP25: 0.81
AP50: 0.74
AP70: 0.54
AP75: 0.46

Mask AP: (wo/ CRF)

AP25: 0.74
AP50: 0.44
AP70: 0.16
AP75: 0.11

Could you please share your Mask AP result? We have at least 9 points gap compared to the author's reported results.