skyplane
skyplane copied to clipboard
🔥 Blazing fast bulk data transfers between any cloud 🔥
I'm able to access a bucket (in region 'us-east-1') and am trying to move it to another account which I have access to but do not down. However I get...
Setup: Start a long running transfer with `skyplane cp`. Then, delete some objects in the portal midway through the transfer. Once the post-transfer verification completes, the following error is printed:...
Reported by @romilbhardwaj See trace at: https://gist.github.com/romilbhardwaj/7b8b84cf30952d7cf7965d7ec9b3d7ee
We've been manually creating buckets and uploading synthetic data for testing. We should create configurable scripts to generate and clean up the data to save time and eliminate storage overhead.
Many public buckets are hosted as "requester pays" where the user will pay for any associated egress fees. This should be a simple fix to add support for transfers from...
Fixes #486 The upload_test_file.py is a script that creates of the bucket and uploads random objects. It takes 3 arguments: number of files (-n), amount of total bytes in MB...
Updating Readme to say that Skyplane doesn't work without WSL
I ran a transfer with the command ``` skyplane cp s3://sarah-skylark-us-west-1/big_file.txt/big_file.txt s3://sarah-skylark-us-east-1/big_file.txt --multipart ``` ``` Traceback (most recent call last) ────────────────────────────────╮ │ /Users/sarahwooders/.pyenv/versions/3.8.8/lib/python3.8/site-packages/skyplane/cli/cli.py:157 in │ │ cp │ │ │...
`$ skyplane replicate-random gcp:us-central1-a aws:sa-east-1 -n 8 -s $((32)) --chunk-size-mb 4 --debug` client.log: ``` existing rule for 0.0.0.0/0:22 in skyplane, not adding again 17:29:15 [DEBUG] [wait_for] Waiting fn= completed in...
Solution is to attempt to provision gateways in the first zone, then the second, etc. We do this for AWS but not for GCP. Currently, we always default to provisioning...