skyplane icon indicating copy to clipboard operation
skyplane copied to clipboard

[request] System to allow users to spin up individual dockers to join as gateways for transfers

Open ShishirPatil opened this issue 3 years ago • 0 comments

Problem to be solved
For some use-cases, especially for on-prem, skyplane may not be able to launch gateway instances - reasons include permissions, lack to visibility into IP addresses, etc. In such cases we need the ability for the user to launch skyplane dockers on their instances and have these dockers join the pool. A skyplane docker becomes the base unit for transfer.

Clean slate solution
Step 1: A one-step setup where we install skyplane-gateway requirements on each instance. Questions - Do we assume we have sudo permissions? Step 2: Auto check to ensure the firewalls allow connections with external IPs. Step 3: All the dockers must register with one client to be collected into one pool. Step 4: We need to profile if all the dockers have the same capacity - network ingress and egress? Step 5: Hand over the transfer to skyplane for rest of transfer.

Step 6: Allow dockers to join and leave the transfers midway.

ShishirPatil avatar Feb 07 '23 19:02 ShishirPatil