pbnjay
Memory restricted containers on Linux
Both LXC and Docker containers seem to be reporting host system memory instead of available memory. See:
https://github.com/timescale/timescaledb-tune/issues/58
This package uses standard linux system calls, so if LXC and Docker are reporting unexpected values it isn't really something in scope here unfortunately. It sounds like there's a discrepency between the values in /proc/meminfo and Sysinfo_t is that correct?
Yes, looks like sysinfo_t returns host parameters, it doesn't count cgroup mem limit.
Upd. My bad, cgroup limit should be available in sys/fs/cgroup/memory/memory.limit_in_bytes
Well, this works for me
func sysTotalMemory() uint64 {
in := &syscall.Sysinfo_t{}
err := syscall.Sysinfo(in)
if err != nil {
return 0
}
// If this is a 32-bit system, then these fields are
// uint32 instead of uint64.
// So we always convert to uint64 to match signature.
memTotal := uint64(in.Totalram) * uint64(in.Unit)
// cgroup v1 detection
if limitRaw, err := os.ReadFile("/sys/fs/cgroup/memory/memory.limit_in_bytes"); err == nil {
if limit, err := strconv.ParseUint(strings.TrimSuffix(string(limitRaw), "\n"), 10, 64); err == nil {
if limit < memTotal {
return limit
}
}
}
// cgroup v2 detection
if limitRaw, err := os.ReadFile("/sys/fs/cgroup/memory.max"); err == nil {
if limit, err := strconv.ParseUint(strings.TrimSuffix(string(limitRaw), "\n"), 10, 64); err == nil {
if limit < memTotal {
return limit
}
}
}
return memTotal
}
I thnk, free memory can be calculated as memory.limit_in_bytes - memory.usage_in_bytes or memory.max - memory.current.
For cgroup v2 it looks like we'll need to get the process relative cgroup path from /proc/self/cgroup but this approach should work.
I will want to make this secondary check consistent across platforms, so I'll have to test equivalent code for BSD limits and Windows as well.
If I log into a LXC container and run free or cat /proc/meminfo or top, it will report the 8GB that it is allocated, and not the 128GB of the host. I am surprised it is even aware if how much the host has.
Actually, it seems it's only inside the Docker container that it is the issue.
Host:
❯ cat /proc/meminfo | grep MemTotal
MemTotal: 131920188 kB
LXC:
❯ cat /proc/meminfo | grep MemTotal
MemTotal: 8388608 kB
Docker on host:
❯ docker run --rm --memory=2g ubuntu sh -c 'cat /proc/meminfo | grep MemTotal'
MemTotal: 131920188 kB
Docker inside LXC (this one surprised me the most):
❯ docker run --rm --memory=2g ubuntu sh -c 'cat /proc/meminfo | grep MemTotal'
MemTotal: 131920188 kB