nerdctl
nerdctl copied to clipboard
containerd
Logging drivers enhancement
What is the problem you're trying to solve
Target Nerdctl logging drivers:
- [ ] none
- [x] json-file : https://github.com/containerd/nerdctl/pull/959
- [ ] local
- [x] journald : https://github.com/containerd/nerdctl/pull/1062
- [x] fluentd : https://github.com/containerd/nerdctl/pull/1073
- [ ] fluentbit
- [ ] syslog
Describe the solution you'd like
Configure logging drivers
Additional context
No response
Maybe I can handle the rest of the task in this issue. @fahedouch would you mind this issue to me?
thank you for you request @Zheaoli . I start writing code for journald so let me finish it based on your refacto https://github.com/containerd/nerdctl/pull/1050 . You can enhance the rest of the list
@fahedouch Great! Maybe I can write the fluentd and syslog first. BTW, maybe we need spilt the logger in two parts: the readable logger and the unreadable logger
@Zheaoli What do you mean by readable logger and the unreadable logger here ?
Yes, because some of the loggers such as fluentd can not read the log directly after we wrote the log. So I think maybe we need to split the logger by two types
Probably we should support using multiple drivers simultaneously. Then we can read/write jsonfiles while writing to fluentd.
Probably we should support using multiple drivers simultaneously. Then we can read/write jsonfiles while writing to fluentd.
SGTM.
Probably we should support using multiple drivers simultaneously. Then we can read/write jsonfiles while writing to fluentd.
We are creating a dependency between drivers here; json-file driver issue will impact fluentd driver from nerdctl point of view. This can be voluntary when the user configures generated jsonfiles as fluentd tail input (https://docs.fluentd.org/input/tail)
I think we should implement fluentd driver as a separated LoggerFunc, we can implement one of the fluentd input
using this module https://github.com/fluent/fluent-logger-golang
I have a question, should we make the log driver 100% compatible with Docker? Some of the drivers are easy to implement but hard to E2E tests, such as AWS Cloudwatch log driver or others. @fahedouch @AkihiroSuda
I think the AWS CloudWatch driver should be implemented as a third party plugin that can be called with --log-driver binary://foo
I think the AWS CloudWatch driver should be implemented as a third party plugin that can be called with --log-driver binary://foo
SGTM, I think I can make a design proposal
I would like to try working on fluentbit driver if that is fine and no one is working on it.
@containerd/nerdctl-maintainers
Are we fine to add the fluentbit driver as a built-in driver, or should it be an external driver (--log-driver binary://foo) ?
@AkihiroSuda I would say external driver; fluentbit driver(s) are be built as go plugin https://docs.fluentbit.io/manual/development/golang-output-plugins#build-a-go-plugin which can be easily handled as external driver. Also fluentbit have multiple deps so I think it is better to isolate it as external plugin.
@manugupt1 Yes you can start working on this. I started implementing the local driver which also an external plugin. We just have to agree on the structuring of external plugins
@fahedouch do you have a draft PR for local that I can take inspiration from for external driver?
@manugupt1 is there any update about this issue?
Sorry, been busy with work a lot. So, I have not made much progress
@AkihiroSuda @fahedouch @Zheaoli Should we close this as third party support is open or keep it open for inbuilt drivers?
Yeah would be great to read the logs using directly nerdctl :-)
nerdctl -n k8s.io container logs e780b0e8aba7
WARN[0000] Currently, nerdctl logs only supports containers created with nerdctl run -d
FATAL[0000] open /var/lib/nerdctl/1935db59/containers/e780b0e8aba78200dfe6112dc834925952ee6bb3a39f64cb8ab88081c77c74f9/log-config.json: no such file or directory
Hello Everyone,
I am Rishabh, an undergraduate computer engineering student. I am interested in this issue. Is it still open?
Best Regards Rishabh Singh
Hello Everyone,
I am Rishabh, an undergraduate computer engineering student. I am interested in this issue. Is it still open?
Best Regards Rishabh Singh
Hi Rishabh, Yes, please go ahead. What lnterest you in this ticket ?
Hello Everyone, I am Rishabh, an undergraduate computer engineering student. I am interested in this issue. Is it still open? Best Regards Rishabh Singh
Hi Rishabh, Yes, please go ahead. What lnterest you in this ticket ?
- [ ] local
- [ ] fluentbit
- [ ] syslog
I guess, These three logging drivers are not implemented yet.
Hi @fahedouch, can this ticket also include adding support for "none" logging driver?
The use case is to be able to disable container logs from compose file.
e.g. nerdctl compose up of docker-compose.yml
version: "3.7"
services:
compute:
image: public.ecr.aws/ubuntu/ubuntu:22.04
logging:
driver: "none"
results in:
INFO[0000] Ensuring image public.ecr.aws/ubuntu/ubuntu:22.04
INFO[0000] Re-creating container repro-compute-1
FATA[0000] unknown logging driver "none": not found
FATA[0000] error while creating container repro-compute-1: exit status 1
References: [1] https://docs.docker.com/config/containers/logging/configure/#supported-logging-drivers
@fahedouch are you working on the local log driver? Do you have a draft for the design for inspiration :). I can help with the implementation.
@vsiravar unfortunately I didn't have enough time to make a clear enough design. The idea is to separate the local driver into a different binary then consume it using containerd runtime v2 logging plugin mode.
"binary://BIN?KEY=VALUE" URI is parsed into Args {BIN, KEY, VALUE}.