twemcache icon indicating copy to clipboard operation
twemcache copied to clipboard

Add metrics to track memory/heap consumption

Open thinkingfish opened this issue 12 years ago • 0 comments

Learning about the real memory consumption of Twemcache is important to correctly estimate overhead and avoid paging. To many people's surprise, slab memory doesn't account for the entire heap size in many cases, and it would be helpful to have metrics reflecting actual heap size and its composition.

Aside from slabs, large memcache instances usually allocate a lot of memory into hashtable(s); and for instances with a lot of connections, connection buffer is also a significant source of memory overhead. So it would be nice to have the following metrics for starters:

heap_curr /* total heap size, everything allocated through mc_*alloc */
heap_hashtable /* size of the current hashtable, and if in transition, hashtables */
heap_conn /* connection buffer related overhead */

There are others that could be added, such as slab size (which can currently be computed from slab_curr and slab size), suffix buffer for reply messages, etc. It would be nice to come up a more comprehensive component list, but they probably aren't as important as the above ones.

thinkingfish avatar Jan 14 '13 21:01 thinkingfish