argobots
argobots copied to clipboard
high memory consumption for producer-consumer layouts with default settings
If one ES consistently launches ULTs to execute on a different slave ES (for example, to offload specific calculations or I/O operations), then the per-ES stack queue on the latter will eventually fill up and consume hundreds of MiB of RAM. If that ES never launches it's own ULTs then those entries will not be used and the queue will never be emptied.
The workaround is to set the ABT_MEM_MAX_NUM_STACKS environment variable to a low value (it defaults to 65536) to limit how many entries will be saved in the per-ES queue. This is done automatically in the margo library: https://xgitlab.cels.anl.gov/sds/margo/blob/master/src/margo.c#L195
See https://lists.argobots.org/pipermail/discuss/2018-February/000020.html for discussion, just entering on github so that we don't lose track of the issue.