Discussion: CacheService eviction size computation review

[rigazilla]

The operator tries to dimension the CacheService cache in order to use all the available free memory. This is done setting the cache eviction size. Computation is like this:

Hard coded constants

	consts.CacheServiceFixedMemoryXmxMb            = 200  // javaopt -Xmx
	consts.CacheServiceJvmNativeMb                 = 220  // ???
	consts.CacheServiceJvmNativePercentageOverhead = 1

Fomulas

        // containerMaxMemory: memory allocated for the pod
        // nativeMemoryOverhead: an estimation of the memory overhead needed
	nativeMemoryOverhead := containerMaxMemory * (consts.CacheServiceJvmNativePercentageOverhead / 100) // this needs fix
        // evictTotalMemoryBytes: eviction size, final output
	evictTotalMemoryBytes :=
		containerMaxMemory -
			(consts.CacheServiceJvmNativeMb * 1024 * 1024) -
			(consts.CacheServiceFixedMemoryXmxMb * 1024 * 1024) -
			nativeMemoryOverhead

I'd like to hear opinions on the overall computation and especially on the constants values.

[dmvolod]

@rigazilla it also get unpredictable results, as all calculations are performing against node, where podList.Items[0] is scheduling. This means in case of pod re-scheduling or any others moving or decreasing number of nodes. We should define an algorithm which is not related on whole node start options. For example, user defined values or calculated with formulas, and if we can fit with these values, go next, if not, try to reduce amount of requested memory and try to fit again.

[rigazilla]

Current computation seems to use this model:

cachesize = totalMemory-(memoryOS+maxJavaHeapSizeXmx) and then use cachesize in the cache template:

            <memory>
                <off-heap 
                    size="` + cachesize + `"
                    eviction="MEMORY"
                    strategy="REMOVE"
                />
            </memory>

is this an adequate model?

Code now has the following values for computation

        totalMemory = 512  (default value)
	maxJavaHeapSizeXmx  = 200  // javaopt -Xmx (consts.CacheServiceFixedMemoryXmxMb)
	memoryOS = 220                             (consts.CacheServiceJvmNativeMb)

Do we really need 440Mb for a clear startup?

[wburns]

cachesize = totalMemory-(memoryOS+maxJavaHeapSizeXmx) sounds good to me.

However 200 MB for JVM heap seems a bit much if we only have a single off heap cache.

Also ISPN 12 shouldn't require as much OS memory as the # of threads should be substantially lower than before.

[rigazilla]

I think we can plan two tasks:

• simplify the current model (nativeMemoryOverhead is something for fine tuning not needed now)
• do some memory usage tests to better tune the memoryOS and maxJavaHeapSizeXmx parameters both for Ispn 11 and 12