container runtime is failing to store logs in json format after switching from docker to containerd on eks 1.24

[rootrahulagr]

We have recently started using containerd as container runtime, since eks 1.24 has ended support for dockershim. Some of our applications are failing now as they are expecting logs in json format.

With docker, we had this configuration in /etc/docker/daemon.json -

{   "bridge": "none",   "log-driver": "json-file",   "log-opts": {     "max-size": "250m",     "max-file": "10"   },   "live-restore": true,   "max-concurrent-downloads": 10,   "default-ulimits": {     "memlock": {       "Hard": -1,       "Name": "memlock",       "Soft": -1     }   }       So far, I couldn't find any way to do a similar configuration for containerd. Could someone please advise how to achieve this with containerd runtime?   TIA! 

[HenryXie1]

We hit same issues,we need log format to be json. per doc https://github.com/containerd/containerd/blob/main/docs/man/containerd-config.toml.5.md it is only for debug sock.

[skadem07]

@HenryXie1 did you find any solution, we also have similar config. i think we can specify json format format (Default: "text") sets log format. Supported formats are "text" and "json"

how should we use live-restore flag?

[HenryXie1]

I update this in fluentd which fix the issues

@type regexp
expression /^(?<time>.+) (?<stream>stdout|stderr)( (?<logtag>.))? (?<log>.*)$/

[wesleyoliveiraleite]

Hi guys, any update about it? I've check doc https://github.com/containerd/containerd/blob/main/docs/man/containerd-config.toml.5.md, but debug mode JSON it's only for logs of systemd containerd, not for logs of containers/pods in /var/log/pods.

I use a filebeat to collect logs from kubernetes pods e Elasticsearch to parse logs JSON, but after changing the container runtime from docker to containerd, it's not working any more, because the format of log it's not in JSON.

Before: Containerd runtime format: 2021-06-22T10:27:25.813799277Z stdout F some log

After: Docker runtime format ( JSON ) : {“log”:“some log\n”,“stream”:“stdout”,“time”:“2021-06-22T10:27:25.813799277Z”}

[pasztorl]

+1 for pod logs in json format

[nadavbuc]

+1 for json log format Regexp Format is working but it's more "expensive" in resources (we use fluentBit and had to tune it a lot to prevent it from losing a lot of the logs) With Json Format the structure is more easily parsed by log collectors