Add slabinfo checks

[treydock]

These checks we've used to detect kernel memory leaks that will cause a node to consume all physical memory but not be detected by conventional means like top, ps or free. We use these checks on our affected system to look for kmalloc in slabinfo that is going to consume large amounts of memory.

* || check_slabinfo_num_objs -m 'kmalloc-*' 5000000
* || check_slabinfo_active_objs -m 'kmalloc-*' 5000000
* || check_slabinfo_active_slabs -m 'kmalloc-*' 5000000
* || check_slabinfo_num_slabs -m 'kmalloc-*' 5000000

[treydock]

Just in case someone comes across this the checks we're now using are basically marking node offline is kmalloc item in slabinfo is at >5GB based on active object count and size of each object:

* || check_slabinfo_active_objs -m 'kmalloc-8' 671088640
* || check_slabinfo_active_objs -m 'kmalloc-16' 335544320
* || check_slabinfo_active_objs -m 'kmalloc-32' 167772160
* || check_slabinfo_active_objs -m 'kmalloc-64' 83886080
* || check_slabinfo_active_objs -m 'kmalloc-96' 55924053
* || check_slabinfo_active_objs -m 'kmalloc-128' 41943040
* || check_slabinfo_active_objs -m 'kmalloc-192' 27962026
* || check_slabinfo_active_objs -m 'kmalloc-256' 20971520
* || check_slabinfo_active_objs -m 'kmalloc-512' 10485760
* || check_slabinfo_active_objs -m 'kmalloc-1024' 5242880
* || check_slabinfo_active_objs -m 'kmalloc-2048' 2621440
* || check_slabinfo_active_objs -m 'kmalloc-4096' 1310720
* || check_slabinfo_active_objs -m 'kmalloc-8192' 655360

[mej]

I'd like to get a 1.4.3 release out the door soon(ish), and since the current dev branch has had quite a lengthy life in production at LANL (and likely elsewhere), I feel safer sticking with it for the release and targetting new development at 1.4.4. (Don't worry; 1.4.4 won't take nearly as long to get out the door as 1.4.3 did, now that we've gotten the legal hurdles dealt with!)

Does that make sense?

[treydock]

Ya 1.4.4 sounds fine, we have the check deployed locally and gives us a bit of time to run this in production to refine if needed.