skyplane

skyplane copied to clipboard

[perf] Add socket timeouts to the gateway sender

1

For sending data along sockets, we should set timeouts on the gateway sender to cancel pending requests, restart the sender/receiver pair and then requeue any pending chunks. This would help...

parasj

Skyplane dependencies should be partitioned by cloud

1

At the moment, if a user only needs to use AWS or GCP, they will also need to install the significant dependencies required for Azure. Ideally, Skyplane dependencies can be...

parasj

``` $ skyplane init ... (3) Configuring GCP: Do you want to configure GCP support in Skyplane? [Y/n]: * GCP region config missing! GCP will be reconfigured. * GCP credentials...

parasj

The object store call `obj_store_interface.initiate_multipart_upload(dest_obj)` is not parallelized and is a bottleneck for large numbers of chunks per object. The calls initiate the multipart uploads should be parallelized.

sarahwooders

Currently, the API is complex in architecture due to mixing of provisioning logic with the dataplane construction. This PR extracts provisioning logic into a separate class for easier testing and...

parasj

This PR creates a draft interface for managing transfers using a programmatic interface. See demo.py: ```python client = SkyplaneClient(aws_config=AWSAuthenticationConfig()) dp = client.direct_dataplane("aws", "us-east-1", "aws", "us-east-2", n_vms=1) with dp.auto_deprovision(): dp.provision() #...

parasj

sarahwooders

sarahwooders

`skyplane init` error: `ModuleNotFoundError: No module named 'azure'`

7

I pip installed Skyplane and ran `skyplane init` but immediately got an error: ``` (base) [ec2-user@ip-172-31-80-140 ~]$ skyplane init Traceback (most recent call last): File "/opt/conda/bin/skyplane", line 5, in from...

sarahwooders

Move GCP networking + pricing logic out of cloud provider

2

Changes: - [x] Move pricing logic to GCPPricing - [x] Move GCP networking logic to GCPNetwork - [x] Shift firewall rules to apply to a single VPC more cleanly -...

parasj