canonical
lxd libraries don't expose facilities to set log level
Required information
- Distribution: Ubuntu
- Distribution version: 22.04
- The output of "lxc info" or if that fails:
- Kernel version: 6.5.0-14-generic
- LXC version: 5.20
- LXD version: 5.20
- Storage backend in use: Any
Issue description
The LXD libraries don't expose any facility to set (or get) the log level of its logger implementation. LXD uses logrus' logger internally, which implements these interfaces, but it's not possible to access them either because they don't export the underlying logger. As such, any project that uses LXD's libraries and wants to set the log level needs to resort to rather ugly hacks.
Steps to reproduce
Importing the LXD libraries and trying to access the logrus logger results in an error since it isn't exported. LXD's libraries don't have any interfaces to control the log level either.
Sorry about the confusion. I mean these: https://github.com/canonical/lxd/tree/main/shared/logger
Please can you explain a bit more about what you're trying to do? Thanks
Yes, of course.
The microceph project currently leverages LXD libraries to simplify deployment and make use of a comprehensive code base. One issue we've seen in embedded deployments is the fact that storage may be constrained quite a bit and so we'd like to control the log level to circumvent such restrictions. However, the LXD logger doesn't provide us with any way to do this and so we had to resort to a rather ugly hack to get around this limitation. See here for more details.
@masnax please can you review this and come up with suggestion(s) on how we should proceed with logging in micro* packages.
@masnax @lmlg perhaps micro* repositories should switch to using slog from Go https://go.dev/blog/slog
@masnax @lmlg perhaps micro* repositories should switch to using slog from Go https://go.dev/blog/slog
I personally don't think a library switch is needed. In our case, it would suffice to have interfaces to get/set the log level (something that the logrus logger already does).
I think it's a good idea to allow dynamically changing the log level just for the added flexibility in LXD. LXD does have lxc monitor but setting the log level to debug for the daemon itself still requires a restart so this could help with that.
As for no longer relying on LXD for logging in microcluster projects, it would be a bit of a re-implementation with a more modern package but it would be a good litmus test for eventually using slog with LXD.
I believe slog requires go 1.21 so if these projects still need to support an earlier go version for an LTS track or something, then they'd likely still rely on LXD's logger, so adding the capability here makes sense.
So I think the approach here will be to add the capability of setting the log level in LXD, but then also moving upstream microcluster to slog.
They are all only distributed by snaps so we control the Go version, 1.21 should be fine.
@masnax any reason not to just switch to slog and avoid needing to dependent on LXD's logger (we will probably switch to slog too eventually), but for something as general as a logger I'm not seeing the value in depending on LXD's one, am I missing something?
@masnax any reason not to just switch to slog and avoid needing to dependent on LXD's logger (we will probably switch to slog too eventually), but for something as general as a logger I'm not seeing the value in depending on LXD's one, am I missing something?
To clarify, I am going to move microcluster to slog. I just also think having this in LXD is useful for LXD.
The initlal reason we used LXD's logger was to avoid reimplementing one for each LXD-adjacent project. All the micro-apps were designed with LXD in mind, so using the same utilities keeps the code looking consistent, and keeps the output consistently formatted too.
The initlal reason we used LXD's logger was to avoid reimplementing one for each LXD-adjacent project. All the micro-apps were designed with LXD in mind, so using the same utilities keeps the code looking consistent, and keeps the output consistently formatted too.
Yeah I agree, I expect we'll move to slog too.
@roosterfish @skozina shall we close this or move it to microcluster repo? I believe the plan is to move to Go's built in slog and not depend on LXD's logger for micro* right?
shall we close this or move it to microcluster repo
Agree, let's move it over.
Hello folks,
It would be great if in addition you could expose possibility to configure log output format or maybe refactor Microcluster to accept custom logger that would conform to a custom interface (can be inspired by slog)
In MAAS we are using zerolog with a custom format and send everything to systemd journal and at the very end out log looks like this:
Mar 18 10:16:05 maas maas-agent[128763]: INF Started Worker Namespace=default TaskQueue=r38367@agent:main WorkerID=r38367@agent:128763
However Microcluster (that we use as a package) brings a different format:
Mar 18 09:59:23 maas maas-agent[19050]: time="2025-03-18T09:59:23Z" level=debug msg="Dqlite connected outbound" local="127.0.0.1:46996" remote="127.0.0.1:5280"
@troyanov we will most likely move to slog so it may be necessary to write a formatter for slog to match what you have for zerolog, but sounds good :)
@markylaing are you planning to use a global logger, inject it into the context or just pass it explicitly as a dependency?
If you intend to use a global logger via slog.SetDefault, it feels that it doesn't make much sense to initialize the logger here: https://github.com/canonical/microcluster/blob/79df0501b2e96419b7effb371422cd2b563b9590/microcluster/app.go#L71-L73
Instead, the logger should be initialized in main only? https://github.com/canonical/microcluster/blob/79df0501b2e96419b7effb371422cd2b563b9590/example/cmd/microd/main.go#L36-L40
Hi @troyanov, we haven't been able to get around to this one but I expect we will allow a logger to be set via the DaemonArgs, and made available in handlers via state.State or via context.
@markylaing thanks!
The reason I am asking is because I thought I can help with this move (I just did a similar refactoring in MAAS), however looking at the codebase it is not very clear how would you like to handle situation where existing logging is done through a global logger:
https://github.com/canonical/microcluster/blob/79df0501b2e96419b7effb371422cd2b563b9590/internal/endpoints/util.go#L32 https://github.com/canonical/microcluster/blob/79df0501b2e96419b7effb371422cd2b563b9590/internal/recover/recover.go#L558