aws-sdk-go-v2 icon indicating copy to clipboard operation
aws-sdk-go-v2 copied to clipboard

Add UploadedBytes to UploadOutput in feature/s3/manager

Open shawnHartsell opened this issue 4 years ago • 2 comments

Is your feature request related to a problem? Please describe.

I'm migrating a service from https://github.com/minio/minio-go to the V2 SDK and using the s3Manager to handle uploads (the service is essentially an HTTP proxy that sites in front S3 where clients can stream uploads). Since we're uploading using an io.Reader (from the http request body) we don't know the size of the content being uploaded. For auditing and monitoring purposes, it would be useful if the UploadOutput struct returned this data.

Describe the solution you'd like

A field is added to UploadOutput that shows the total upload size of content

Describe alternatives you've considered

Currently, I'm working around the issue by using a custom Build middleware that's injected for each call to Upload that tracks the content-length for single and multi-part uploads (by inspecting the x-id query parameter in the http.Request).

Long term, we've also considered event based solutions such as creating a Lambda to trigger via s3:ObjectCreated:* events from S3, or creating a DynamoDB stream (our content meta-data index) that preforms a HeadObject call when an item is marked as uploaded.

shawnHartsell avatar Aug 30 '21 21:08 shawnHartsell

I imagine we could also do what we needed if https://github.com/aws/aws-sdk-go-v2/issues/941 is implemented as well

shawnHartsell avatar Aug 30 '21 21:08 shawnHartsell

Hi @shawnHartsell , Sorry for the lack of response, but yea this is something we are thinking of implementing just haven't got to it yet.

KaibaLopez avatar Dec 17 '21 22:12 KaibaLopez

@shawnHartsell -- as you've indicated, #941 will cover this and that's the route we intend to take. Closing as duplicate.

lucix-aws avatar Oct 18 '23 15:10 lucix-aws

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

github-actions[bot] avatar Oct 18 '23 15:10 github-actions[bot]