promscale icon indicating copy to clipboard operation
promscale copied to clipboard

Better defaults for cache sizes

Open niksajakovljevic opened this issue 2 years ago • 3 comments

To achieve good performance we are currently recommending our users to tweak following:

--metrics.cache.series.initial-size=30000000
--cache.memory-target=90% 

Let's make sure that cache memory defaults work well enough out of the box. Cache tweaks should be only used for extreme cases. We should validate our findings/improvements by running benchmarks.

niksajakovljevic avatar Oct 24 '22 09:10 niksajakovljevic

Fixes in https://github.com/timescale/promscale/pull/1705 mean that great majority of users shouldn't have to tweak metrics.cache.series.initial-size . Benchmark numbers look pretty good now with default settings so cache.memory-target can stay the same.

niksajakovljevic avatar Nov 29 '22 09:11 niksajakovljevic

Reopening issue to validate existing DefaultSeriesCacheSize = 250000 setting which seems low.

niksajakovljevic avatar Nov 29 '22 14:11 niksajakovljevic

Base on memory heap profile existing DefaultSeriesCacheSize = 250000 uses around 70MB. Maybe we should reconsider increasing it to 500K or maybe even 1M? (140MB or 280MB)? That would make defaults more appropriate for realistic data loads and would mean less overhead of growing cache. @VineethReddy02 What are your thoughts?

niksajakovljevic avatar Dec 05 '22 18:12 niksajakovljevic