skyplane

skyplane copied to clipboard

Although Skyplane can transfer bucket folders using the `-r` flag, it cannot match prefixes with regex to transfer subsets of a folder (e.g. `s3://bucket/imagenet/train_*`). The AWS CLI supports filtering based...

sarahwooders

Airflow has operators for transferring data across buckets (described in this [post](https://medium.com/apache-airflow/generic-airflow-transfers-made-easy-5fe8e5e7d2c2)).

sarahwooders

`skyplane deprovision` is broken for Azure

1

Any transfer involving Azure with `--reuse-gateways` followed by `skyplane deprovision` will leave orphaned resources. Skyplane deprovision should clean up VPNs, etc.

ShishirPatil

Better handling of instance capacity errors

1

Can we re-try instance requests with a smaller number of instances if it fails? Currently I get this error trying to run a transfer: ``` Error running provision_gateway_instance: An error...

sarahwooders

replicate_random() should support same region

1

Given the current evolution of skyplane, we should make replicate_random between src==dest to move data between 2 machines within the same region. https://github.com/skyplane-project/skyplane/blob/b5e06b7cac1a1d6c737d448666666d805316608b/skyplane/cli/cli_internal.py#L39

ShishirPatil

Ctrl-C during a deprovision operation should ask the user for confirmation to exit

4

~Need a way to interrupt `skyplane cp` transfers mid-way by the user.~ We currently prevent users from exiting during a deprovision operation to avoid leaked VMs. A Ctrl-C during deprovision...

ShishirPatil

I have a case where AWS account A has read permissions to S3 bucket A, and is trying to write to S3 bucket B owned by AWS account B with...

sarahwooders

[WIP] Skyplane problems in WSL2 For Windows Users

2

This is an ongoing and **work in progress** list of potential problems of WSL2 that I faced when using Skyplane or Skyplane-related problems. - Problem with some scripts. I wasn't...

HaileyJang

Glacier has very similar boto3 APIs to S3: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glacier.html. We can add support for directly interfacing with Glacier by creating a `GlacierInterface` object store connector similar with the `S3Interface` (https://github.com/skyplane-project/skyplane/blob/main/skyplane/obj_store/s3_interface.py)....

sarahwooders

[RFC] Skyplane API Design Draft

3

# Example usage of the Skyplane API Continued from: https://codeshare.io/K8YNX8 ## Example usage with simple copy API ``` python # inline API that automatically manages the session for a single...

abiswal2001