kic-reference-architectures icon indicating copy to clipboard operation
kic-reference-architectures copied to clipboard

bug: filebeat default path does not work for all configurations

Open qdzlug opened this issue 2 years ago • 2 comments

Describe the bug When running in EKS, the default filebeat path (/var/lib/docker/containers) works fine. However, other deployments use different paths - for example, RKE2 uses /var/log/containers.

Need to make this configurable.

To Reproduce Steps to reproduce the behavior:

  1. Deploy to RKE or microk8s.
  2. Watch as no logs get scraped.

Expected behavior We need to make this configurable; I don't know that we can automatically guess what we're running on, so this may just be up to the user (or handled as part of the configuration process).

Your environment

  • n/a

Additional context n/a

qdzlug avatar Dec 21 '21 15:12 qdzlug

Since we are seeing more deployments on K3s, Microk8s, and Minikube this is becoming more of a problem. Looking into some other options to manage this from the OS side (ie, doing a symlink) which may be slightly easier than trying to detect and configure appropriately.

qdzlug avatar May 06 '22 20:05 qdzlug

Relevant code is here: https://github.com/nginxinc/kic-reference-architectures/blob/master/pulumi/python/kubernetes/logagent/main.py#L68-L73

Default path is /var/lib/docker/containers/${data.kubernetes.container.id}/*.log\noutput.elasticsearch:\n",

qdzlug avatar May 06 '22 20:05 qdzlug