kafka-connect-storage-cloud
kafka-connect-storage-cloud copied to clipboard
S3 Object Tagging
Supporting S3 Object Tagging would be helpful for organization, bookkeeping and monitoring.
Adding a configuration option for S3 Object Tags would be preferred. Another option would be to allow additional HTTP header configuration (enabling users to set tags via the x-amz-tagging
header).
Thanks, Brandt
Supporting initially a small set of the configuration options that are available through the S3 SDK was intentional.
This seems useful. I'm marking this issue as "enhancement". Tags are defined per object, at a maximum number of 10 tags. Do you have a specific use case that you'd like to share regarding tagging?
@kkonstantine thanks for your reply!
I'm running Kafka Connect in distributed mode against multiple Kafka topics. I'd like to be able to:
- Tag objects with the name of the topic that they were created from.
- Tag all objects that were created with Kafka Connect.
This would enable cost analysis and CloudWatch monitoring/alerting by object tag.
FYI, this was merged: https://github.com/confluentinc/kafka-connect-storage-cloud/issues/68 (tags for start and end offsets as well as record count)
Any updates ? We would like to have functionality to add specific tags to be able to setup lifecycle police in S3 based on tags.
Like hardcoded key:value pairs from configuration
We need tags aswell, preferably customisable from headers.
This issue should be solved by my PR above. Is there anything else to be done to close it?