Mirror So2Sat and implement download functionality

[calebrob6]

trafficstars

Currently, the So2Sat dataset can't be auto downloaded because torchvision's download_url doesn't seem to support FTP URLs like "ftp://m1483140:[email protected]/training.h5".

The So2Sat authors are okay with having the dataset mirrored on a website like https://zenodo.org/. Once the dataset has been mirrored we can implement the download functionality as normal.

(@mehmetgunturkun this would be a great contribution if you have the time)

[adamjstewart]

Note that this also applies to SEN12MS. @calebrob6 did the SEN12MS authors give us permission for SEN12MS too?

[calebrob6]

I haven't asked, however SEN12MS is 510GB so it wouldn't be trivial to upload to Zenodo (we'd have to split it up), and it would take a longg time to download over http. Maybe it is better to make users understand what they are getting into in that case?

For reference:

• The official So2Sat repo is here https://github.com/zhu-xlab/So2Sat-LCZ42
• The official SEN12MS repo is here https://github.com/schmitt-muc/SEN12MS

[mehmetgunturkun]

Yes of course, it was the main reason why I asked! I'll take a look

[adamjstewart]

@wangyi111 and I talked about uploading these datasets to HF.