Disk free limit and memory high water mark "relative" configuration always takes precedence over "absolute" configuration

[Zerpet]

Description

The behaviour in configuration key precedence is not consistent for certain configuration keys. Specifically, they configuration key pair disk_free_limit.relative + disk_free_limit.absolute, and the pair vm_memory_high_watermark.relative + vm_memory_high_watermark.absolute. The *.relative variants seem to take precedence over their *.absolute counterparts.

In a setup with a configuration file rabbitmq.conf with the key disk_free_limit.relative = 1.0, and a configuration file conf.d/user-configuration.conf with disk_free_limit.absolute = 2GB, I would expect the absolute disk free limit to take precedence, because it was set "later" in the configuration.

The use case for setting both configuration keys at the same time is to use rabbitmq.conf as a "vendor" configuration, or vendor defaults; then let the user customise the configuration via conf.d/*.conf files. For other keys, they take precedence when set in conf.d/*.conf.

[michaelklishin]

Neither was designed to be used in combination with the other.

[gtato]

If you use the rabbitmq operator with bitnami images you are exactly in the situation @Zerpet described. I also would expect the later config to take precedence.

[kinoute]

I encounter the same problem, especially with k8s & Bitnami images as @gtato stated. In this image, there is a default setting disk_free_limit.relative = 1.0 therefore everything added after that such as disk_free_limit.absolute = 2GB is ignored.

[mleclercq]

I faced the same issue. As a workaround I set the RABBITMQ_DISK_FREE_ABSOLUTE_LIMIT env-var on the rabbitmq container. That env-var is used by the entry-point of the image to set the disk_free_limit.absolute.

To do this I added the following to the RabbitmqCluster resource:

spec:
  override:
    statefulSet:
      spec:
        template:
          spec:
            containers:
            - name: rabbitmq
              env:
              - name: RABBITMQ_DISK_FREE_ABSOLUTE_LIMIT
                value: 1GB

[SimonUnge]

Hi. I'm giving this a look to see what can be done if any.

[SimonUnge]

I did some digging around.

To change the the precedence order, i.e absolute > relative, is doable. But, we'd just 'change' the problem, not solve it.

I assume the proper solution here is to give absolute and relative the same precedence, and the 'later' the config parameter is read will win, i.e the last 'read' version of disk_free_limit will be the winning one, regardless if it is absolute or relative.

We use cuttlefish to read the config files, and while cuttlefish do honer the config item/file order when it comes to individual item values (a newly read value replaces an earlier read one), it does not honor prefix order.

We would most likely require a minor change in the cuttlefish lib as well to solve this neatly. @michaelklishin

[michaelklishin]

It makes more sense for the absolute value to take precedence. It's true that when both are configured, the node is given mixed messages.

@SimonUnge perhaps I am missing why we need to change Cuttlefish to order the keys in a certain way. Would it not be sufficient for RabbitMQ itself, after it'd read the configuration, to evaluate both settings and, if both are present, log a warning and use the absolute one?

[SimonUnge]

Sorry, I'll clarify!

Yes, to just give absolute precedence will be a minor and easy change I can provide, with a warning if both values are configured.

I was thinking though, that it would be preferred to have the two options, absolute and relative, of equal precedence, and which ever is loaded last, wins. I.e if you have disk_free_limit.relative in 01.conf file, and disk_free_limit.absolute in 02.conf file (thus, 02.conf file will be parsed after 01.conf), the value we would use would be disk_free_limit.absolute, since it was the 'last read value', thus taking precedence.

However, the way cuttlefish replaces/updates values internally, prohibits me from making such a distinction easily, as cuttlefish 'updates' its proplist (not delete and replace) and thus it keeps the order:

replace_proplist_value(Key, Value, Proplist) ->
    lists:keystore(Key, 1, Proplist, {Key, Value}).

Making it pretty hard for us to distinguish the order it read the to values.

If cuttlefish would instead remove the old entry, and append the new one at the tail, we could use the order of the values to figure out which value should take precedence.

But, if we are happy with just absolute taking precedence, I can make that change quickly.