ACE_TAO icon indicating copy to clipboard operation
ACE_TAO copied to clipboard

ACE_Dev_Poll_Reactor avoid handle arrays and Handler_Repository size based on RLIMIT_NOFILE

Open nfrmtkr opened this issue 5 months ago • 0 comments

The ACE_Dev_Poll_Reactor initialized handle arrays and a Handler_Repository based on a given size or ACE::max_handles (). The ACE documentation states:

On Unix platforms, the size parameter should be as large as the maximum number of file descriptors allowed for a given process. This is necessary since a file descriptor is used to directly index the array of event handlers maintained by the Reactor's handler repository. Direct indexing is used for efficiency reasons. If the size parameter is less than the process maximum, the process maximum > will be decreased in order to prevent potential access violations.

In standard Linux installation like Redhat or RockyLinux 7/8/9 the "max. open files" limit is usually 1024. Running those distributions in a docker container (e.g. redhat/ubi8) this limit is set to 1.048.576 by default. This adds ~20MB memory usage per ACE_Dev_Poll_Reactor instance. In container based environment like Kubernetes the memory footprint of processes has to be kept low as possible, especially if the PODs should scale up.

A workaround is to execute ulimit -n 1024 in a docker entrypoint script before starting the main process.

The epoll_create() documentation says that the size argument is obsolete:

the kernel dynamically sizes the required data structures without needing the hint

From epoll point of view the size is no longer needed, however the data structure used by ACE_Dev_Poll_Reactor is based on it.

Chaning the recactor internal management avoiding RLIMIT_NOFILE based size would be a valuable improvement.

nfrmtkr avatar Jan 26 '24 09:01 nfrmtkr