skyplane icon indicating copy to clipboard operation
skyplane copied to clipboard

Directly upload/download from AWS Glacier with a `GlacierInterface` object store connector

Open sarahwooders opened this issue 1 year ago • 0 comments

Glacier has very similar boto3 APIs to S3: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glacier.html. We can add support for directly interfacing with Glacier by creating a GlacierInterface object store connector similar with the S3Interface (https://github.com/skyplane-project/skyplane/blob/main/skyplane/obj_store/s3_interface.py).

Requirements:

  • Add --storage-class GLACIER CLI flag to specify (for AWS-only) if the transfer should interface with Glacier or S3.
  • Implement GlacierInterface with boto3 API

Questions:

  • Should Glacier only be supported as a destination, nor a source? How can we transfer data with a Glacier source?

sarahwooders avatar Sep 28 '22 21:09 sarahwooders