Attempt to raise the soft limit to get more file descriptors under systemd

[ekohl]

I'm forwarding this issue from https://bugzilla.redhat.com/show_bug.cgi?id=2132774.

The environment is that Mosquitto 2.0.14 is started via systemd on RHEL8.

It's observed that at most 1015 connections are accepted by default, but if a systemd override with LimitNOFILE is made it can be increased further. This surprised me because systemd on EL8 by default should give a hard limit of 262144 but has a soft limit of 1024. The 1015 connections are surprisingly close to 1024. The 9 other ones can probably be accounted for by stdin/stdout/stderr and some other files like configs.

https://github.com/systemd/systemd/blob/23f3a6f5ff864fd26063c6c35fdaa6d85de566c7/NEWS#L5880-L5914 does have quite a bit to say on this.

Note that the soft limit remains at 1024 for compatibility reasons: the traditional UNIX select() call cannot deal with file descriptors >= 1024 and increasing the soft limit globally might thus result in programs unexpectedly allocating a high file descriptor and thus failing abnormally when attempting to use it with select() (of course, programs shouldn't use select() anymore, and prefer poll()/epoll, but the call unfortunately remains undeservedly popular at this time).

Then later:

Programs that want to take benefit of the increased limit have to "opt-in" into high file descriptors explicitly by raising their soft limit.

So my question is: should Mosquitto attempt to raise its soft limit to, by default, get many more file descriptors or is this unsafe due to select() usage?

[rswindell]

select() will still have the 1024 socket limit (at least on Linux): /usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024

Mosquitto should be changed to use poll() instead.

[ekohl]

To be clear: I don't know if mosquitto uses select(). I'm not a C programmer and approaching this from a user point of view.

[ekohl]

Someone pointed me to https://github.com/eclipse/mosquitto/pull/1884 which may solve it.

[pablomh]

While we wait for the long term solution, could we add an override systemd unit file like the one shown in the BZ?

I'm testing right now one with LimitNOFILE=infinity that after https://github.com/systemd/systemd/pull/5795 will default its value to the max open files set by the kernel.