SC4S "sc4s_recv_time" setting creates lots of small buckets due to strings.data filling with epoch timestamps

[RKH-splunk]

Logging this on behalf of customer.

cisco_asa logs being sent by SC4S to Splunk results in lots of small buckets being generated for the destination index. The warning below is reported by Splunk in the logs :

"The percentage of small buckets (100%) created over the last hour is high and exceeded the red thresholds (50%) for index=cisco_asa, and possibly more indexes, on this indexer. At the time this alert fired, total buckets created=39, small buckets=39"

Further investigation discovered that the strings.data file within each bucket was filling up with epoch timestamps due to the use of "sc4s_recv_time". Un-setting this in the conf file below resolved the issue :

#/opt/sc4s/local/config/filters/app-postfilter-drop_metadata.conf block parser app-postfilter-cisco_asa_metadata() { channel { rewrite { unset(value('fields.sc4s_recv_time')); }; }; }; application app-postfilter-cisco_asa_metadata[sc4s-postfilter] { filter { 'cisco' eq "${fields.sc4s_vendor}" and 'asa' eq "${fields.sc4s_product}" }; parser { app-postfilter-cisco_asa_metadata(); }; };

This needs a permanent update/change to make this the default.

[rjha-splunk]

We will check the feasibility in version 3 as we already provided work around for this.

[mattweber78]

Just curious, what is the workaround for this issue?

[rjha-splunk]

Its mentioned in the issue itself @mattweber78

[mattweber78]

Ok thanks. Thought there was another workaround without having to disable that field.

[rjha-splunk]

no this field is coming out of the box , so to stop creating small buckets based on this field the only way i foresee is dropping the field.