FreeMask icon indicating copy to clipboard operation
FreeMask copied to clipboard

failure to reproduce accuracy with provided data and code

Open HongkLin opened this issue 6 months ago • 6 comments

Thank you for your work. However, I encountered a problem in the process of reproduction. I repeated the experiment several times using the data and code provided by you, but I could not reproduce the results in the paper. Specifically, I trained a segformer-b4 model using the ade synthetic data provided by you and the real data according to your code. Training four times, we get 50.1, 49.5, 49.7, 49.8, but we can't reach the 50.6 in the paper. Even using mask2former-t with mixed training following your code is only 50.8. Not only did we mix real data, but even when we trained segformer-b4 on synthetic data, we only got 47.9. Is the synthetic data you provided consistent with the experiment in the paper? Or maybe some Settings in the code didn't reset after a few trials.

HongkLin avatar Jul 29 '24 08:07 HongkLin