argobots icon indicating copy to clipboard operation
argobots copied to clipboard

high memory consumption for producer-consumer layouts with default settings

Open carns opened this issue 6 years ago • 0 comments

If one ES consistently launches ULTs to execute on a different slave ES (for example, to offload specific calculations or I/O operations), then the per-ES stack queue on the latter will eventually fill up and consume hundreds of MiB of RAM. If that ES never launches it's own ULTs then those entries will not be used and the queue will never be emptied.

The workaround is to set the ABT_MEM_MAX_NUM_STACKS environment variable to a low value (it defaults to 65536) to limit how many entries will be saved in the per-ES queue. This is done automatically in the margo library: https://xgitlab.cels.anl.gov/sds/margo/blob/master/src/margo.c#L195

See https://lists.argobots.org/pipermail/discuss/2018-February/000020.html for discussion, just entering on github so that we don't lose track of the issue.

carns avatar Apr 24 '18 13:04 carns