Tracker: All logs to container stdout
Description
All logs generated by GCS components must go to the container stdout so they can be picked up by the cluster logging infrastructure.
New GCS components such as gd2 and csi drivers are already sending their logs appropriately, so this item is targeted mainly at older gluster components. Specifically:
- FUSE mount logs
- Parent of gluster/gluster-csi-driver#3
- Brick logs
- self-heal daemon
- (any other process that normally writes into /var/log/gluster)
The current proposal is to run a sidecar container that contains rsyslog and use that to collect the logs and present them to stdout, most likely with some format modifications such that individual log streams are discernible. The majority (all?) of the above items have the ability to send to rsyslog.
Once this is implemented, we should no longer be writing log files to storage (ephemeral or otherwise) within a GCS related pod. All logging should go through the cluster logging infrastructure by way of container output.
Related items
(As we create individual issues to track work, they can be added here)
cc: @ShyamsundarR @julienlim
Reducing scope for 1.0 because dependent projects lack dev bandwidth.