How to increase net.code.somaxconn ?

[yaroslav-nakonechnikov]

trafficstars

hello, we see this:

this looks a bit strange and we would like to understand how it would be possible to increase the limits?

this is splunk container on kubernetes engine.

[vivekr-splunk]

To increase the number of TCP connections within a container, you may need to adjust several kernel parameters and container resource limits. Here are steps to help you achieve this:

1. Increase somaxconn in the Host Cluster Node

The somaxconn parameter determines the maximum number of connections that can be queued for acceptance. You already have somaxconn set to 4096, but you may want to increase this further on the host:

sudo sysctl -w net.core.somaxconn=65535

You can make this change permanent by adding it to /etc/sysctl.conf:

echo "net.core.somaxconn=65535" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

2. Increase Container-Specific Limits

Depending on the container runtime, the container may inherit the host's somaxconn setting. However, it can also have its own limits:

For Splunk Pod You can set sysctl parameters at the Pod level using the securityContext in your Pod manifest:

apiVersion: v1
kind: Pod
metadata:
  name: splunk
spec:
  securityContext:
    sysctls:
      - name: net.core.somaxconn
        value: "65535"
  containers:
    - name: your-container-name
      image: your-image

Since you can't pass the securityContext directly to the Splunk pod through the Splunk Operator custom resource at the moment, you might consider these alternative approaches:

Alternative Approaches Apply Settings on the Host: If feasible, you could set the necessary kernel parameters at the host level (if you control the host machines), which the containers inherit. This wouldn't provide per-container granularity but would solve the immediate need to increase connection limits.

we will work on enhancing Custom Resource to take secruitycontext , thank you

[yaroslav-nakonechnikov]

so, there is no setting for it, and need to create pre-task playbook to make it work, right?

ps. making setting on host is not applied to pods in kubernetes. it should be allowed explicitly:

• https://bottlerocket.dev/en/os/1.20.x/api/settings/kubernetes/#allowed-unsafe-sysctls
• https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/

[yaroslav-nakonechnikov]

@akondur

[splunk@splunk-site1-prod-indexer-0 splunk]$ sysctl -a | grep wmem
sysctl: permission denied on key 'kernel.cad_pid'
sysctl: permission denied on key 'kernel.usermodehelper.bset'
sysctl: permission denied on key 'kernel.usermodehelper.inheritable'
sysctl: permission denied on key 'net.ipv4.tcp_fastopen_key'
sysctl: permission denied on key 'net.ipv6.conf.all.stable_secret'
net.ipv4.tcp_wmem = 4096        20480   4194304
net.ipv4.udp_wmem_min = 4096
sysctl: permission denied on key 'net.ipv6.conf.default.stable_secret'
sysctl: permission denied on key 'net.ipv6.conf.eth0.stable_secret'
sysctl: permission denied on key 'net.ipv6.conf.lo.stable_secret'
sysctl: permission denied on key 'vm.mmap_rnd_bits'
vm.lowmem_reserve_ratio = 256   256     32      0       0
sysctl: permission denied on key 'vm.mmap_rnd_compat_bits'
sysctl: permission denied on key 'vm.stat_refresh'
[splunk@splunk-site1-prod-indexer-0 splunk]$ sudo sysctl -a | grep wmem

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for splunk:



Sorry, try again.
[sudo] password for splunk:
^Csudo: 2 incorrect password attempts

which password should be used for splunk user?