torchgeo icon indicating copy to clipboard operation
torchgeo copied to clipboard

Skip downloading when not in pre-sampled coords

Open tatsubori opened this issue 1 year ago • 3 comments

SSL4EO's download_ssl4eo.py can fail with NoKey error if given pre-sampled coords like sampled_locations.csv do not contain any index specified by the indices range.

tatsubori avatar Jan 23 '24 11:01 tatsubori

@wangyi111 may be interested in reviewing this

adamjstewart avatar Jan 24 '24 16:01 adamjstewart

yes I think if we skip then a warning is good. Normally, the indices range for download_ssl4eo.py is supposed to be covered by sampled_locations.csv. So the warning can help the user check again if this is really needed.

One rare use case could be you want to filter out some coords in sampled_locations.csv (e.g. you want to download more images within an area), move those entries to another sampled_locations_special.csv file for download. In this case, you still specify the full indices range in sampled_locations.csv to download_ssl4eo.py, and the script can skip those ids that are not in the new csv.

wangyi111 avatar Jan 25 '24 11:01 wangyi111

I'd recommend returning False if skipped and returning True if completed, then at the end of the script printing how many were skipped.

calebrob6 avatar Jan 26 '24 13:01 calebrob6

I added a warning message. Adding return values and summing them is more work, but @calebrob6 can add another commit/PR if he wants.

adamjstewart avatar Feb 29 '24 14:02 adamjstewart