hls-foundation-os icon indicating copy to clipboard operation
hls-foundation-os copied to clipboard

Published 20 hours ago • verified publisher icon NASA-IMPACT

Validation vs. Test set for the burn scars dataset

Open gabrieltseng opened this issue 2 years ago • 1 comments

Hello!

Thank you for releasing these datasets.

I understand that you use the "validation set" to report test results in the paper. It also seems like you use this set as a "true" validation set:

https://github.com/NASA-IMPACT/hls-foundation-os/blob/df7761efdcedd187372b1c19bde113e2040bfd5b/configs/burn_scars.py#L141

Did you use the "val/test" set for both validation / hyperparameter selection and reporting test results?

If not, what split did you use for validation & hyperparameter selection?

Thanks!

gabrieltseng avatar Nov 13 '23 15:11 gabrieltseng

In addition, would it be possible to share the actual values (for all metrics, in addition to IoU for the wildfire scar class) used to make the plots in Table 10 of the paper?

gabrieltseng avatar Nov 14 '23 12:11 gabrieltseng