skypilot
skypilot copied to clipboard
[Storage] file mount with object store download the data again by `sky launch` twice
sky launch twice the following yaml will download the data again. This will cause unnecessary costs for downloading. Maybe we should consider using aws s3 sync instead of aws s3 cp to do it incrementally.
resources:
cloud: AWS
accelerators: V100:8
file_mounts:
/data/imagenet.tar.gz: s3://sky-imagenet-tar/sky-imagenet.tar.gz
setup: |
cd /data
tar xvzf /data/imagenet.tar
cd -
git clone https://github.com/pytorch/examples