fluent-plugin-s3
fluent-plugin-s3 copied to clipboard
fluent
Amazon S3 input and output plugin for Fluentd
original: https://github.com/fluent/fluent-plugin-s3/pull/188 related: https://github.com/fluent/fluent-plugin-s3/pull/285 > ## PR **it will be greatful that the plugin can push log to owner S3 Storage like Ceph S3.**
With AWS SDK, it is possible to set the property [Configuration#user_agent_prefix](https://docs.aws.amazon.com/AWSRubySDK/latest/AWS/Core/Configuration.html#user_agent_prefix-instance_method) which will be used for every request against AWS services. Currently, the project I'm working on needs to set...
### Changes made: - Allowing multiple s3 bucket names in the input s3 plugin and preserving the previous functionality with a single bucket name - Added dynamic fetching of objects...
Hey! I was trying to create buckets with time slices but the plugin only creates the bucket (if needed) at start. I made some changes and my proposal is to...
Hi everyone, Is possible set the bucket name with the name of a record field, or tag? Example ``` @type record_transformer enable_ruby true auto_typecast true tenant ${record["attrs"]["tag"].split('|')[1]} (...) @type copy...
AWS s3 object has customizable metadata, the aws CLI version of getting the custom metadata is located here: https://docs.aws.amazon.com/cli/latest/reference/s3api/head-object.html While I think the bucket & key is useful but it...
I believe the amazon ruby SDK supports client side encryption on S3 buckets. Is that something that could be added here? https://aws.amazon.com/blogs/developer/using-client-side-encryption-for-s3-in-the-aws-sdk-for-ruby/
I'm testing fluent-plugin-s3 v1.5.1 (The previous version is v1.2.1) I got the following warning message on boot. > #0 [user-identification-user-s3] No ${chunk_id} or %{uuid_flush} in s3_object_key_format with multiple flush threads....
I am running fluentd in a Kubernetes cluster. When configuring the `s3_out` plugin, instead of providing the `access_key` and the `secret_key`, I used the ``. My s3 configuration looks like...
I added a new compression plugin based on facebook zstandard compression system. zstandard offers faster and better compression then gzip ``` bash $ time zstd -f -5 random.log -o random.log.zst...
