[FR] select the logger implementation without rebuilding

[fujitatomoya]

Feature request

Feature description

Currently there is no way to select the logging backend without building rcl. This is because that when building rcl, we must choose the logging implementation to be used.

In default rcl_logging_spdlog is selected if RCL_LOGGING_IMPLEMENTATION is not set. But we can change it with RCL_LOGGING_IMPLEMENTATION and available logging backend library.

e.g rcl_logging_noop

export RCL_LOGGING_IMPLEMENTATION=rcl_logging_noop
colcon build --symlink-install --cmake-clean-cache --packages-select rcl_logging_noop rcl

and, no log files are generated with rcl_logging_noop

root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 run demo_nodes_cpp talker
[INFO] [1724349615.489636579] [talker]: Publishing: 'Hello World: 1'
[INFO] [1724349616.489588912] [talker]: Publishing: 'Hello World: 2'
[INFO] [1724349617.489610598] [talker]: Publishing: 'Hello World: 3'
[INFO] [1724349618.489512697] [talker]: Publishing: 'Hello World: 4'
...
root@tomoyafujita:~/.ros/log# ls

This generates the constraint that whole system needs to use only specified logging backend in this workspace. It would be nice if we can build the logger backend libraries, and select the logging backend when the program starts.

Implementation considerations

Current related code is,

https://github.com/ros2/rcl/blob/dcfe9ba4f95e3c0d546d30b3f03b769df16667a1/rcl/cmake/get_default_rcl_logging_implementation.cmake#L26-L34

Probably we can escalate RCL_LOGGING_IMPLEMENTATION like how RMW_IMPLEMENTATION works to select the rmw implementation when executing programs.

[clalancette]

I'll state upfront that I'm not against this feature.

But are there any practical uses for it? While I think that our logging subsystem has a number of problems, swapping out the disk logging backend at runtime doesn't seem to be one of them. So I'd like to hear about a practical use before we spend time on it.

[fujitatomoya]

@clalancette

the use case (not production yet) that i have off the top of my head, to support 2 (or multiple) logger backend in the same system.

• one could be default spdlog, that just stores the logging files in specific file system. (on-system log just in case.)
• we have proprietary in-memory logger backend, some performance critical application uses. (once the memory chunk is full, squash those into the file system and send it to the cloud or something.)
• not all, but some nodes are system application, so that we want to send the log data via logging data pipeline (such as rsyslog, FluentBit) to entire system observability to collect and aggregate the log data, and give the feedback to the system.

note: i am spending a couple of days for rcl_logging_syslog (forwarded to FluentBit), so i can share more details about this. I think i can disclose the source code with some documentation in this weekend. (or hopefully today 😄 )

[fujitatomoya]

@clalancette i developed some PoC based on syslog and support FluentBit, please see overview https://github.com/fujitatomoya/rcl_logging_syslog

[ros-discourse]

This issue has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-log-system-meets-rsyslog-and-fluentbit/39280/3

[fujitatomoya]

I am not sure if i can make this into Kilted, but had quick code scan to comprehend what needs to be done.

• introduce rcl_logging_implementation package in https://github.com/ros2/rcl_logging to abstract the logging implementation instead of dynamically built with rcl, so that rcl can only depend on rcl_logging_implementation and rcl_logging_interfaces without any implementation details.
• rcl_logging_implementation should be implemented similar with https://github.com/ros2/rmw_implementation, the main focus of this package is to resolve all the rcl logging interface symbols, support dynamic loading the user specified implementation and connect symbols to them internally.
• rcl probably should have init function in the process space to call rcl_logging_indentifier_check that makes sure it fails if user specifying unavailable logging implementation. (similarly https://github.com/ros2/rcl/blob/1427bbb73af6bda69036e5efe1255dfb90329186/rcl/src/rcl/rmw_implementation_identifier_check.c#L172-L178)
• rcl should always use default rcl_logging_spdlog if not specified RCL_LOGGGING_IMPLEMENTATION env variable at runtime.

I think we need a basic design document to describe more details before implementation.

[ciandonovan]

I'll state upfront that I'm not against this feature.

But are there any practical uses for it? While I think that our logging subsystem has a number of problems, swapping out the disk logging backend at runtime doesn't seem to be one of them. So I'd like to hear about a practical use before we spend time on it.

~~It would be the only way users could disable logs being written to disk while keeping their stdout logs without recompiling rcl (by switching it to the noop backend)~~

[clalancette]

It would be the only way users could disable logs being written to disk while keeping their stdout logs without recompiling rcl (by switching it to the noop backend)

As I've mentioned elsewhere, that is not the case. We already have --disable-external-lib-logs available. While I understand you don't like the UI for that (and we can discuss that elsewhere), that use case doesn't require this feature in my opinion.

[fujitatomoya]

besides https://github.com/ros2/rcl/issues/1178#issuecomment-2644237076,

in general, this decoupling should be much better.

• this allows user to plugin their own logging implementation without any source build of rcl. some users do not have their own CI/CD pipeline, in that case this can be burden.
• similar to above situation, we have own CI/CD pipeline to build the entire ros2 packages as base system. the logging implementation can be different based on the production use case, that says individual business units use different logging implementation. in this situation, we need to build the rcl package for each use cases. i do not think this is ideal.

[fujitatomoya]

@Barry-Xu-2018 can you review design doc?

https://docs.google.com/document/d/1HLXqzbx7AeSpZgqksyUMM_9zDwWLrSvrIRt8rN-FoyE/edit?tab=t.0

[Barry-Xu-2018]

@Barry-Xu-2018 can you review design doc?

https://docs.google.com/document/d/1HLXqzbx7AeSpZgqksyUMM_9zDwWLrSvrIRt8rN-FoyE/edit?tab=t.0

Okay.

[Barry-Xu-2018]

@fujitatomoya

https://docs.google.com/document/d/1HLXqzbx7AeSpZgqksyUMM_9zDwWLrSvrIRt8rN-FoyE/edit?tab=t.0

LGTM. I do have a minor point I’d like to get your thoughts on.
The variable RCL_LOGGING_IMPLEMENTATION is used both at compile time and at runtime. Can we clearly separate its usage?
During compilation, a logging dynamic load library (such as rcl_logging_dyn_load which is implemented in https://github.com/ros2/rcl_logging) is build if RCL_LOGGING_IMPLEMENTATION is empty.
At runtime, a new environment variable LOGGING_IMPLEMENTATION is used to specify the final logging library to be used.

[fujitatomoya]

The variable RCL_LOGGING_IMPLEMENTATION is used both at compile time and at runtime. Can we clearly separate its usage?

i think it describes the clear difference between them in google doc RCL_LOGGING_IMPLEMENTATION environment variable what is your unclear point on this?

At runtime, a new environment variable LOGGING_IMPLEMENTATION is used to specify the final logging library to be used.

There will be no such environment varibale LOGGING_IMPLEMENTATION, but only RCL_LOGGING_IMPLEMENTATION as designed?

[Barry-Xu-2018]

The variable RCL_LOGGING_IMPLEMENTATION is used both at compile time and at runtime. Can we clearly separate its usage?

i think it describes the clear difference between them in google doc RCL_LOGGING_IMPLEMENTATION environment variable what is your unclear point on this?

Yes. The description is very clear. I'm just a bit concerned that using a variable at both compile time and runtime might cause confusion for users.

At runtime, a new environment variable LOGGING_IMPLEMENTATION is used to specify the final logging library to be used.

There will be no such environment varibale LOGGING_IMPLEMENTATION, but only RCL_LOGGING_IMPLEMENTATION as designed?

I just want to separate the variables used during compile time and runtime, which is why I proposed introducing this variable. Of course, this will result in the introduction of a new environment variable.

This is just one of my thoughts. I don't have a strong inclination to make changes to the current design.

[fujitatomoya]

@Barry-Xu-2018 ah, i see. what i am trying to do with RCL_LOGGING_IMPLEMENTATION is similar with RMW_IMPLEMENTATION which is also used effectively in build and runtime. if we specify RMW_IMPLEMENTATION at build time, it will statically link to the specified RMW implementation only.

[Barry-Xu-2018]

what i am trying to do with RCL_LOGGING_IMPLEMENTATION is similar with RMW_IMPLEMENTATION which is also used effectively in build and runtime. if we specify RMW_IMPLEMENTATION at build time, it will statically link to the specified RMW implementation only.

Yes. RMW_IMPLEMENTATION can also be used at the compilation stage. I overlooked this.