logstash-output-s3
logstash-output-s3 copied to clipboard
Permissions on subfolders
We are trying to use one bucket for multiple applications, but separating them into subfolders. In AWS it is possible to give permissions to subfolders only, but the plugin requires access rights on the root level of the bucket as well. I guess thats because there are some calls where the prefix is not included. It would make sense to always add the prefix if there is one specified to avoid access to the root level.
Figuring this out was a bit of a trial-and-error exercise, and its not really specified anywhere in the documentation or made obvious in the code.
It would be quite helpful to specify the access rights the plugin needs and that it needs it at root level.
The problem here is due to the fact that we try to write a test file at the root of the bucket to validate the credentials when we start logstash.
Bad credentials are really the major problem we encounter when we debug stuff with users, I think we could allow people to disable the check when they have special permission on their bucket.
For others: You can add the resource "arn:aws:s3:::{bucket_name}/logstash-programmatic-access-test-object-*"
to your policy as this is the path that it tries to write to, rather than granting full bucket access.
@ph I think all it would take to solve this issue is to write the test file to the prefix
specified in the config rather than the root of the bucket.
Would you merge a PR that would do this?