Run fluentd with a custom configuration

[adrianabasoc-uipath]

I am trying to run fluentd with a custom configuration instead of the predefined one from https://github.com/microsoft/Docker-Provider/blob/ci_prod/build/linux/installer/conf/kube.conf. In particular, I am interested in changing the run_interval for the kube_events plugin. I was thinking that the configuration from https://github.com/microsoft/Docker-Provider/blob/ci_prod/charts/azuremonitor-containers/templates/ama-logs-rs-configmap.yaml#L5 is the way to go. However, I couldn’t see any place in which that configuration is used, apart from some if-statements. I would like to know what is the purpose of that configmap? And there is any way to change the fluentd configuration? What I did in the end was to change the run_interval directly in the /etc/fluent/kube.conf, but I was thinking there should be a better way to achieve this without any workaround.

[ganga1980]

Hi, @adrianabasoc-uipath , we are working on feature to provide customers to change the data collection settings such as interval and namespaces exclusion for applicable data types such as KubePodInventory, KubeNodeInventory and container inventory etc but not for kube-events. Question, why do you want change the run_interval for kube_events and how does this help?

This ama-logs-rs-configamp not configurable and this needs to be removed from the yaml. if you are interested to try out the cost optimization feature preview, please reach out us on [email protected] with subscription(s) or cluster ids which you want to try.

[github-actions[bot]]

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[adrianabasoc-uipath]

We would like to change the run_interval for kube_events because when getting the data from k8s it uses lists instead of watches, which is not very efficient.

[github-actions[bot]]

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 12 days with no activity.

[ganga1980]

We would like to change the run_interval for kube_events because when getting the data from k8s it uses lists instead of watches, which is not very efficient.

We already implemented watch for the types ( Pod & Node) which were are making high number of APIs call in large cluster. The total number of API calls making from our agent is much smaller what we were making before. We have plan to move to Watch API for remaining resource types as well. changing the run_interval to higher value, will make the events getting lost with list API. is loss of events not an issue for your scenarios?