root icon indicating copy to clipboard operation
root copied to clipboard

[TUnix] RAM stats (gSystem->GetMemInfo) to htop/procps style

Open aaronj0 opened this issue 1 year ago • 7 comments

@vepadulano @dpiparo

This PR fixes

  • https://github.com/root-project/root/issues/7196

This issue is very case-dependent. When trying to calculate memory usage, normally the actual free memory on the system is free + buffer + cached

If shared memory usage is high (e.g mmaping a big cache) the calculation is slightly different: (free + buffer/cache) - shared

The author of htop mentions on this question that htop uses this convention(in 2016):

  • Total used memory = MemTotal - MemFree
  • Non cache/buffer memory = Total used memory - (Buffers + Cached memory)

Now htop (managed by htop-dev) uses this (on https://github.com/htop-dev/htop/blob/main/linux/LinuxMachine.c#L210):

const memory_t usedDiff = freeMem + cachedMem + sreclaimableMem + buffersMem;
host->usedMem = (totalMem >= usedDiff) ? totalMem - usedDiff : totalMem - freeMem;

This issue intends to merge the total used memory to include cached instead of just free which isn't strictly a correct solution.

This PR implements the current approach utilised by htop which includes the following fields to calculate MemUsed, MemAvailable and SwapUsed:

Int_t     fMemAvailable; // available RAM in MB
Int_t     fMemCached; // cached RAM in MB
Int_t     fMemBuffer; // buffer RAM in MB
Int_t     fMemShared; // shared RAM in MB
Int_t     fSwapCached; // cached swap in MB
Int_t     fSReclaimable // slab that might be reclaimed

Results:

➜  ROOT free -h
              total        used        free      shared  buff/cache   available
Mem:           23Gi       3.4Gi       7.4Gi       214Mi        12Gi        19Gi
Swap:         7.6Gi       2.1Gi       5.5Gi
➜  ROOT root -l
root [0] MemInfo_t memInfo; gSystem->GetMemInfo(&memInfo);
root [1] cout << memInfo.fMemTotal << " " << memInfo.fMemUsed << " " << memInfo.fMemFree << endl;
23877 2786 7562
root [2] 

Checklist:

  • [x] tested changes locally

aaronj0 avatar Feb 13 '24 16:02 aaronj0

Can one of the admins verify this patch?

phsft-bot avatar Feb 13 '24 16:02 phsft-bot

Nice, thanks for the PR !! Here a link to a draft PR, in case it inspires you for the analogous on TWinNTSystem: https://github.com/root-project/root/pull/7208

ferdymercury avatar Feb 13 '24 16:02 ferdymercury

@phsft-bot build

hahnjo avatar Feb 13 '24 17:02 hahnjo

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

phsft-bot avatar Feb 13 '24 17:02 phsft-bot

Thanks for this very nice improvement. I like the idea of doing the same of a widely adopted tool. Do you think it's possible to translate this to Macos and, perhaps more difficult, Windows?

dpiparo avatar Feb 14 '24 08:02 dpiparo

Test Results

    10 files      10 suites   1d 16h 24m 43s :stopwatch:  3 019 tests  3 019 :white_check_mark: 0 :zzz: 0 :x: 26 211 runs  26 211 :white_check_mark: 0 :zzz: 0 :x:

Results for commit f0751d25.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Feb 14 '24 10:02 github-actions[bot]

MacOS is currently WIP

aaronj0 avatar Feb 15 '24 13:02 aaronj0

What are the expected values for mac? htop uses used = active + wired (https://github.com/htop-dev/htop/blob/2503239d9fb453b5d67d3b33690c5a8e914bc58c/darwin/Platform.c#L305) while we are currently using used = active + inactive + wired. Should we keep Used as-is and have fMemAvailable = fMemTotal - (active + wired) (i.e. fMemAvailable = inactive + free) ?

silverweed avatar Aug 19 '24 12:08 silverweed