rsyslog 8.35.0: config check rsyslogd -N 1 fails if imfile is loaded without files to monitor

[selivan]

Expected behavior

If imfile is loaded without any files to monitor: rsyslogd -N 1 should print warning, but return code should be success.

Loading imfile without files to monitor is not always a problem. In my case, I load the module in common role for all hosts, and files to monitor are added in roles for particular software. Not all hosts have software that writes log files.

Ability to check config by command exit code is required to prevent ending with broken configuration after updating config with automation software like ansible or puppet.

Actual behavior

root@ubuntu:~# rsyslogd -N 1
rsyslogd: version 8.35.0, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: imfile: no files configured to be monitored - no input will be gathered [v8.35.0 try http://www.rsyslog.com/e/2212 ]
root@ubuntu:~# echo $?
1

Steps to reproduce the behavior

• add module(load="imfile") to rsyslog.conf
• run rsyslogd -N 1
• check exit code

Environment

• rsyslog version: 8.35.0
• platform: Ubuntu 16.04

[norpol]

With out the -N 1 option I'm also getting the warning, but rsyslogd is starting successfully.

[selivan]

@norpol It does start successfuly, which means the config is correct, but it fails config check. That does not play right with automation systems, like ansible/puppet/etc.

[rgerhards]

coming back to here. The question is if the automation system should generate the module load if not needed. But as it looks what we need to "solve" the issue is have a global setting (or command line option) that makes rsyslog ignore warnings during config check and just check for hard errors.

Note that I would consider loading an unneeded module an error for a manually crafted config. It usually is a sign that something was forgotten - maybe the file monitor definition.

[davidelang]

On Fri, 26 Apr 2019, Rainer Gerhards wrote:

Note that I would consider loading an unneeded module an error for a manually crafted config. It usually is a sign that something was forgotten - maybe the file monitor definition.

One common reason to do this is to have the module load in rsyslog.conf so that config snippets that use the module can be dropped into /etc/rsyslog.d without having to try and figure out if a module load needs to be added (or risking duplicate module loads)

David Lang

[selivan]

@rgerhards I think this would be the best solution: -N 1 makes a soft check, returning 0 if rsyslog can start with this config, and -N 2 makes a hard check, returning 0 only if there is no warnings.

Example of multiple module load statement: I have multiple ansible roles that add configs to rsyslog.d for forwarding log files from disk to logserver. Host can have none of this roles, only one of them or several - depends on the host type. If I load imfile in each of them, I will get warning for loading already loaded module on hosts with several roles. If I load imfile in rsyslog.conf, I will get warning for loading unnecessary module on hosts without any of that roles. So if I want to have rsyslog config that passes config check, I have to make some type of global variable indicating if this host needs loading imfile. Not convenient at all.

Other software, like Apache and Nginx, does not return error on loading unnecessary modules. Config check is correct if service can start with this config files, even if warnings about obsolete directives or something else are printed.

[klesher]

I am running into this issue as well, with an identical use case for @selivan (Ansible roles creating rsyslog.d configs).

Would a reasonable alternative be to allow module loads to be repeated, at least within rsyslog.d files? This would allow modules dependencies for includes to be self-contained without need for the main rsyslog.conf file to load the module.

Ex/ rsyslog.d/nginx.conf:

module(load="imfile")

input(
 Type="imfile"
 Tag="nginx/error_log"
 File="/var/log/nginx/error.log"
)

rsyslog.d/mysql.conf:

module(load="imfile")

input(
 Type="imfile"
 Tag="mysql/error_log"
 File="/var/log/mysql/error.log"
)

[rgerhards]

multiple loads are dangerous, as they could specify different module parameters. We could add a parameter along the lines of ignoreIfLoaded="yes" which would do exactly this. Would that help? But again, it's dangerous.

[klesher]

I don't want to speak for others, but I believe that would work for me. Would be nice to also get some input from @selivan if possible since they opened the issue.

We're using a basic module(load="imfile") statement with no parameters to load the module and would be okay with continuing to use that pattern in order to make this more friendly for automation purposes.

[selivan]

@rgerhards yes, ignoreIfLoaded="yes" should do the trick, good idea. It may allow loading without other parameters or with exactly same parameters as before, attempt to load the same module with different parameters should produce an error. This way it would not be dangerous.

[davidelang]

what do you do if the different module loads don't have the same parameters?

[selivan]

@davidelang

what do you do if the different module loads don't have the same parameters?

throw an error, config is incorrect

[rgerhards]

throw an error, config is incorrect

that requires quite some changes; e.g. we need to introduce an interface to do a parameter check. Note that the original design goal was to prevent multiple loads.

[rgerhards]

also... would no parameters given also mean that the defaults need to be used - or does it mean a conflicting set if previously parameters were set? These are the questions we faced with the old config system, and the answer was that we no longer permit such fuzzy constructs.

[selivan]

@rgerhards it is also possible to have a check if module is loaded:

if not module_is_loaded(name='imfile') {
    module(load="imfile")
}

This will solve the problem and does not require to implement multiple times module loading.

[davidelang]

I like this much more than the other options that have been discussed.

David Lang

On Thu, 28 Nov 2019, Selivanov Pavel wrote:

@rgerhards it is also possible to have a check if module is loaded:

if not module_is_loaded(name='imfile') {
   module(load="imfile")
}

This will solve the problem and does not require to implement multiple time module loading.

[pgporada]

I'm running into an ordering issue with my configuration management and think that the idea proposed by @selivan would be very nice to have.

[WimpyMan]

Hi,

Just letting you know that I encountered this exact same problem today... and that I will therefore need to go live without validating my config... Which is a shame.

[klesher]

Hi,

Just letting you know that I encountered this exact same problem today... and that I will therefore need to go live without validating my config... Which is a shame.

Not sure if it will work for your use case, but as a workaround I place a rsyslog.d file on all of my systems to monitor a rarely changing file via imfile (say, /var/log/boot.log).

This allows me to unconditionally include the imfile module regardless of system, and also prevents any issues when running a config validation after future changes.

[WimpyMan]

Hi, Just letting you know that I encountered this exact same problem today... and that I will therefore need to go live without validating my config... Which is a shame.

Not sure if it will work for your use case, but as a workaround I place a rsyslog.d file on all of my systems to monitor a rarely changing file via imfile (say, /var/log/boot.log).

This allows me to unconditionally include the imfile module regardless of system, and also prevents any issues when running a config validation after future changes.

Hi,

That won't do for me as the configuration is shared across multiple teams and I wouldn't want to do such things :)

Thanks anyways.

Cheers

[Mafa7]

Hi, I know it's a long time ago, but has this issue been resolved? @rgerhards, @selivan, is there a solution for it? Personally, I like this solution:

if not module_is_loaded(name='imfile') { module(load="imfile") }

Thanks.

[thanos-k]

Any updates on this issue ? I am having the exact same problem.

We can probably implement some convoluted ansible logic to by pass the problem but still the

if not module_is_loaded

solution is probably the cleanest way to do it while keeping the flexibility of multiple different teams just dropping config files in conf.d