logging-operator
logging-operator copied to clipboard
kube-logging
Google Stackdriver output
Is your feature request related to a problem? Please describe. We use stackdriver as an output for fluentd for a manual deploy currently but would like to move to logging-operator.
Describe the solution you'd like I'd like to use the fluent-plugin-google-cloud in the same manner as the other backend based plugins like ElasticSearch.
Describe alternatives you've considered I have considered a GCS logging bucket and a function to ship the logs from there into the StackDriver API but it is a less than ideal option.
Additional context This was previously raised in #40 but considered not required unless someone was willing to raise it as a separate request later.
Ay,
Is StackDriver output available ? We want to move from our old logging workflow to Banzai Cloud Logging Operator but StackDriver is mandatory for us.
This becomes a requirement for us too. Can we think of rescheduling this task?
@tarokkk would you be able to provide an update at all please? Just so we can assess to wait or use a different tool. Thanks
This issue is undeservedly staled We are revamping the fluentd image building so we are taking a look at this now!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
The problem, why this has not yet been implemented is the complexity it brings into the already complex and big fluentd image.
One idea I have at the moment is to create a base image that includes the base components only without any third party outputs, so that it is small and faster to build. Users of the the operator can than depend on this base image and add their combination of output plugins on top of that, which would work for their specific use case.
The only thing that bothers me in this case is that we cannot easily decide from the logging operator if an output is supported by the image or not, so if someone sees the google stackdriver output configuration could naively assume, that it is supported in the base image and they would be surprised, that they have to build their own.
Any thoughts?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!
Hi @pepov, THanks for taking a look. II'm pretty sure its not possible but a bare base image with no outputs would be quickets, then at runtime install additiona outputs that the end user needs.?
@ekhaydarov we don't have a method to install plugins at runtime, if someone needs more outputs they have to build a custom image or they will be able to run multiple aggregators (through multiple logging resources) and use different images on each.
There is another option which some of the core maintainers would be open to explore, which is implementing an output for the syslogng based aggregator.
There are two possible solutions in that case as well: migrate to the syslog-ng based aggregator or use a separate syslog-ng logging without a fluentbit agent and relay logs designated in stackdriver into that aggregator.
Let me know if any of the above would work for you.
With the introduction of the new multi-stage image build it is now possible to have an image that supports Google Stackdriver only without having to pull the ruby dependency hell into our full image, which is already quite large. Given this, we can finally implement the required API in the Logging Operator as well. In case someone would need an image that contains not just the Stackdriver output they will have the option to base off of our stackdriver image quite easily.