Transition from S3 to Glacier without copy and delete

[nateiam]

As a Herd Administrator, I want data to move from S3 to Glacier without having to copy the data to another bucket and delete it in the origin bucket.

Tech Note:

• Storage class transition currently operates by performing an S3 to S3 copy to a bucket with a 0d Glacier policy
• This story re-implements storage class transition to occur within the same bucket via object tagging and lifecycle policy

Acceptance Criteria

• Storage policy transitions execute according to new technical approach described above.
• Storage unit for the BData cycles through the following during transition ** Enabled - normal status prior to any archiving ** Archiving - status while archiving (tech note: this applies only to S3 object tagging, not AWS async processing that actually transitions object) ** Archived - status once archiving is complete (tech note: again applies to when tagging is complete, AWS async processing could be ongoing after this)
• Storage policy transitions work only for S3_MANAGED Bucket. Other buckets including KMS will be handled as future stories
• Availability and DDL consider archived objects as "NotAvailable" (tech note: minor changes to existing logic that does this)
• Existing rule that limits transition based on BData size is removed
• Existing logic and data around additional Glacier storage is removed, including: ** Operations that move data to GLACIER_MANAGED ** Logic that checks file existence and size after archiving

Infrastructure required: ** Glacier Lifecycle Policy ** Permissions for S3 object tagging

Existing logic that does not need to be modified and does not need to be separately tested

• All Storage Policy Scheduler logic
• Check that archiving is already underway