sexilog icon indicating copy to clipboard operation
sexilog copied to clipboard

disable bloom filter so save memory?

Open shartge opened this issue 9 years ago • 1 comments

The curator documentation http://www.elastic.co/guide/en/elasticsearch/client/curator/current/bloom.html states:

It can sometime make sense to disable bloom filters. For instance, if you are logging into an index per day, and you have thousands of indices, the bloom filters can take up a sizable amount of memory. For most queries you are only interested in recent indices, so you don’t mind CRUD operations on older indices taking slightly longer.

This setting is especially relevant to Logstash and Marvel use-cases. The bloom command allows you to disable the bloom filter cache for indices.

Since SexiLog uses ES 1.3.9 (ES 1.4 no longer enables bloom filters for search) one should use a command like this ...

curator --loglevel CRITICAL bloom indices --all-indices

... to save memory, if I interpret the documentation and several websites on using ES with time series data correctly.

shartge avatar Mar 31 '15 13:03 shartge

we'll check that on live appliances, thanks

rschitz avatar Mar 31 '15 17:03 rschitz