[Feature Request] Add tagging support to s3

[mikehayesuk]

It would be great if we could tag S3 objects while using s3 commands such as "cp" and "sync".

Thanks!

[kyleknap]

Marking as a feature request. So something like adding a --tagging parameter to cp and sync?

[mikehayesuk]

Marking as a feature request.

Thanks!

So something like adding a --tagging parameter to cp and sync?

Yup, it would be great to take advantage of object tagging while using the CLI.

[kyleknap]

Thanks for clarifying. Makes sense to me.

[dinvlad]

+1, our use case is to enforce CLI users to tag their multipart uploads (through s3:RequestObjectTag/<tag-key> policy condition for PutObject, unless it doesn't work that way).

[yvele]

Did someone found a quick and simple workaround?

[dinvlad]

One workaround is to tag objects on the server side after the user uploaded their files. The main limitation to that is eventual consistency, i.e. the uploaded files may not appear in the listing right away, so you'd need to introduce an arbitrary delay before labeling (I'd also check the combined hash of all keys and ETags as a way of ensuring the listing is 'stable').

[yvele]

@dinvlad Yes I was thinking using an AWS Lambda that triggers when an S3 object is put.. But that's a bit overkill for such a basic feature.. 😄

[dinvlad]

@yvele agreed, it would be nice to have it out of the box. However, it does put more burden on the user because now they also need to remember how to apply tags appropriately.

[ASayre]

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

This entry can specifically be found on UserVoice at: https://aws.uservoice.com/forums/598381-aws-command-line-interface/suggestions/33168343--feature-request-add-tagging-support-to-s3

[jamesls]

Based on community feedback, we have decided to return feature requests to GitHub issues.

[eikenb]

You can tag from the cli using aws s3api put-object-tagging .... I'm currently using it in combo with aws s3 mv ... to migrate some data w/ tags. It works, but I'd rather have it in one call so that if I need to interrupt the script I wouldn't have to worry about something missing the tags.

[bpo]

PR #2890 with boto/s3transfer#96 adds the support for tagging without a second call.

[AbhiPrasad]

Is this still on the table? Would be super helpful to our current workflows!

cc @osman

[trav-c]

I'd like to see this added too, it would simplify parts of our deployment workflow quite a bit.

[NoahSDS]

I want this, too!

[mwhouser]

+1 for an all-in-one "copy and tag".

Without simultaneous tagging, tagging after performing a aws s3 cp . s3://bucket --recursive is difficult.

[ckane]

Have you tried:

aws s3api put-object --bucket bucket-name --key key-name \
  --tagging "Tag_Name=Tag_Value,tag2_name=Tag2_Value" --body filename.ext

[mwhouser]

@ckane Using aws s3api put-object would lose all advantages of using aws s3 cp:

• Multiple file uploads
• Multiple simultaneous uploads, with number and bandwidth throttling
• Many others

[mwkorver]

Noting here that when testing s3 cp and s3api copy-object against small and large files, with existing tagsets, the default behavior is that the tagset gets copied only for small files not large. Guessing that, much like with metadata, the object's tagset is not copied when multi-part is used.

[jsunsch]

Upvote! This would be an excellent feature, as it would greatly enhance my experience with using Lifecycle rules on S3 objects.

[johntdyer]

Wow. I cant believe this is not supported... Please add this

[jefftucker]

This would be extremely useful and would help me be better able to accurately copy objects between buckets (and frequently between accounts) if I could just set a flag that says to also copy the tags from the source object. Here's how I would expect this to work:

• if the source object has no tags, the flag is basically ignored and no error is reported if the file is otherwise copied successfully
• if the destination object exists and has no tags, the source object would be copied over the destination object along with the source object's tags.
• if the destination object has the same tags but different values than the source object, the source object would be copied over the destination object and the tag values for the source object would overwrite the values on the destination object.
• if the destination object has different tags than the source object, I would expect the source object to be copied over the destination object and the destination object would have its tags set to exactly match those of the source object (e.g. a tag present on the destination object but not on the source object would effectively be deleted). I could live with different behaviors than this, but this is what I would expect by default.
• if an error occurs such that the object was copied but the tags were not able to be copied, I would expect a different return value from the command along with an error message stating that the file was copied but the tags were not, or some other simple method of detecting this state. If this means that the tags are in an undefined state (some of them were copied, others were not) then that would be fine with me as well. I'd rather have the file be copied and remain that way so I can just go back and fix the tags than have to re-copy the whole object first and then set the tags.
• other possible error behavior could be that if the bucket is versioned and the object was copied but the tags could not be due to an error would be that the object would become a non-current version and this error would be reported in a way that I could differentiate it from a failure to copy the object itself. That would allow me to go back and restore the non-current version and then manually copy just the tags, so I still don't have to re-copy the whole file. As long as this was clearly documented as behavior that ONLY occurs if versioning is enabled, then I'd be fine with it.
• It would be nice to add this functionality to the aws s3 sync command also, having the same flag and behaviors as the s3 cp command.

[willkara]

Bumping, @kyleknap any ideas for timeline on this one?

It'd great a good idea to add this to cut down on the back-and-forth if we want to add tags to new files.

[vz10]

Hi all,

We're working on this feature and would like to hear back from you about the expected functionality.

Option 1

The first option is to be consistent with other S3 APIs. The value supplied to --tagging will overwrite all the tags. For example if you have the object s3://some-bucket/some-object with the tag "Key1:Value1" and run the command:

aws s3 cp s3://some-bucket/some-object s3://another-bucket --tagging Key2=Value2 --copy-props default

The resulting object will be s3://another-bucket/some-object with the tag "Key2:Value2". Note that the original Key1 is no longer present.

Option 2

Another option is to add an additional parameter (like --merge-tagging) for a strategy to merge tags. This would allow for tags to be either overwritten like in the previous example or merged. For example if you have an object s3://some-bucket/some-object with the tag "Key1:Value1" and run the command:

aws s3 cp s3://some-bucket/some-object s3://another-bucket --tagging Key2=Value2 --copy-props default --merge-tagging

The resulting object s3://another-bucket/some-object will have tags "Key1:Value1,Key2:Value2".

Please let us know what do you think in a comment or just add your reaction under this comment:

• :+1: - Option 1: overwrite tags will be enough
• 🚀 - Option 2: add merge option as well

Thanks!

[willkara]

@vz10 thanks for continuing to work this!

[mwhouser]

@vz10

I see 4 possible options because I see uses for all:

Option 1: New tags only, don't copy from original object

Useful when you don't care what's on the original objects, you just want the copies to have a certain set of tags.

Example:

Object: key1:value1
Command Line: key2:value2
Final Result: key2:value2

Option 2: Add & overwrite -- Copy tags from original object, adding new tags from command line, command line overrides original (if present in both)

Useful when you want to copy all the tags, but you want certain tags to have certain values.

Example:

Object: key1:value1,key2:value2
Command Line: key2:value3,key3:value4
Final Result: key1:value1,key2:value3,key3:value4

Option 3: Add missing only -- Copy tags from original object, adding new tags from command line, but only when missing from the original

Useful if you want to fill in gaps in your tags (populate values for missing tags).

Example:

Object: key1:value1,key2:value2
Command Line: key2:value3,key3:value4
Final Result: key1:value1,key2:value2,key3:value4

Option 4: Change tag values only -- Copy tags from original object, changing tag values only if they exist

Useful if you want to change a bunch of tag values, if they exist, but you don't want them added if they weren't in the original.

Example:

Object: key1:value1,key2:value2
Command Line: key2:value3,key3:value4
Final Result: key1:value1,key2:value3

I also see more options where it could select the tags to copy from the original but that would be overkill I think 😸

[NoahSDS]

I agree with @mwhouser. All of those options would be helpful.

[vortexing]

In our hands, @vz10 both of your options really are required, otherwise it's still an underpowered feature and maybe not worth working on.

As to @mwhouser 's suggestions, I would say Option 1 and 2 are required, and I agree that options 3 (and maybe 4) would be likely next steps.

[BradJaffke]

Are there still plans for s3 sync to have tagging? That would help us drastically!

[ReidWeb]

Per the above mention in the aws-cdk issues, customers utilising @aws-cdk/s3-deployment cannot apply tags through their deployment due to the absence of tagging support on this CLI command, which is an issue when they are utilising permissions boundaries.