Instance-Shadow-Diffusion icon indicating copy to clipboard operation
Instance-Shadow-Diffusion copied to clipboard
Published 7 months ago • verified publisher icon MKFMIKU

The issue with the evaluation script.

Open ZHUXIUJINChris opened this issue 1 year ago • 4 comments

May I ask which script you used for the evaluation? I used the evaluation script from ShadowFormer to assess your ISTD dataset results, but the discrepancy is quite large.

ZHUXIUJINChris avatar Sep 29 '24 09:09 ZHUXIUJINChris

I feel like you might have uploaded the wrong results for ISTD, possibly uploading the AISTD results instead, as there is a significant color difference in the images.

ZHUXIUJINChris avatar Sep 29 '24 09:09 ZHUXIUJINChris

Hi @ZHUXIUJINChris , we rewrote the evaluation script with pure PyTorch and ensured it fully matches the typical Matlab script https://github.com/MKFMIKU/Instance-Shadow-Diffusion/tree/main/evaluation_scripts.

MKFMIKU avatar Sep 30 '24 14:09 MKFMIKU

@ZHUXIUJINChris I am not sure what do you mean by color difference? I just checked the uploaded results and believe they are different images.

MKFMIKU avatar Sep 30 '24 14:09 MKFMIKU

I suspect that you might have uploaded the results of ISTD as the results of AISTD, because the colors in the ISTD results are consistent, whereas in AISTD, due to color adjustments, there are color differences in the same background. Your ISTD reslut set, It looks like the results are from AISTD. e9b96c93a516ffcfa7d0f9b13f1c5be ISTD target b8eca4c1c8cadb9692fd0e00eceab03

ZHUXIUJINChris avatar Oct 17 '24 08:10 ZHUXIUJINChris