logging-operator
logging-operator copied to clipboard
kube-logging
Ability to select namespaces by labels
Is your feature request related to a problem? Please describe. We'd like to be able to define a ClusterFlow that defines an entire subset of namespaces which are labeled. This would allow teams to have a single ClusterFlow instead of potential hundreds of Flows.
As of 3.0 you should already be able to do this! Let us know if it doesn't work for you for some reason. See the clusterflow example here: https://banzaicloud.com/docs/one-eye/logging-operator/log-routing/#match-statement
Hi @pepov, looks like you can list namespaces and and label selectors for pods. Not use label selectors for namespaces, or is that incorrect?
The problem with this approach that there are no namespace labels metadata in the log lines. So in the current architecture, it would be really hard to maintain information about the namespaces and making routing decisions based on namespace metadata. I understand the requirement and it is something I would be glad to see in the future but It would be extremely expensive to implement. In the meantime, you can use all namespaces and have dedicated labels for such purposes. (I know this is a best-effort workaround but I have no other better solution yet)
The correct solution is to mutate the namespace labels to the pod labels. This is because it would require extra RBAC priveleges to query that information and I can't think of a proper way to do it correctly. By that I mean how do you figure out what were the labels on a namespace based on a kubernetes log line that was written in the past?
Let me know if someone has a good idea for a correct implementation.
This is now possible using https://kube-logging.dev/docs/whats-new/#kubernetes-namespace-labels-and-annotations although it requires a few updates in the label router plugin
I see that a pre-release of 4.7 has now appeared - but doesn't include any fixes for this issue. Is the plan still to get this into 4.7 when it releases properly?
FYI I've started to implement and test this:
- https://github.com/kube-logging/fluent-plugin-label-router/pull/21
- https://github.com/kube-logging/logging-operator/pull/1757
the new label router will be available with fluentd image v1.16-4.8 and the whole thing will be part of the 4.8 release scheduled Juli 2nd, but I'm also going to tag a dev image from master soon.