Disc buffer environment variable is multiplied not added

[Jaxjohnny]

https://splunk.github.io/splunk-connect-for-syslog/main/configuration/#sc4s-disk-buffer-configuration

SC4S_DEST_SPLUNK_HEC_DEFAULT_DISKBUFF_DISKBUFSIZE bytes (53687091200) Size of local disk buffer in bytes (default 50 GB)

This creates 10 qf files in the /var/lib/containers/storage/volumes/splunk-sc4s-var/_data folder.

I understand that the total of all the files should be 50GB. However, the 50GB limit is applying to each file.

so with 50GB limit. That means the total space consumption will be 500GB or 10 Threads X 50GB = 500 GB

see my screen shot for details.

[paultek]

Hello,

Is there any workaround I can apply for this? I'm a little stuck on what to do here, our disk is full.

Regards Paul

[rjha-splunk]

Here we need to make the document better to say that it will create each thread of the size mentioned , the disk buffer can be only full if the HEC is not available.

I will increase some size of disk as it can impact the performance of sc4s and can stop it from working altogether, restarting sc4s can help as well, the parameter to control the size of this property is SC4S_DEST_SPLUNK_HEC_DEFAULT_DISKBUFF_DISKBUFSIZE

[jworapong]

The document says "Without disk buffering enabled SC4S can handle up to 345K EPS (800 bytes/event avg) With “Normal” disk buffering enabled SC4S can handle up to 60K EPS (800 bytes/event avg)" That means the SC4S still impact the performance event if the HEC is available. I'm not sure my understanding is correct or not.

[rjha-splunk]

When disk buffering is enabled , it will extra load on SC4S(syslog-ng) to check the destination and created qf files , when they are disabled if the destination is not available it will simply drop after in memory is exhausted.