halo icon indicating copy to clipboard operation
halo copied to clipboard

Running many Halo-enabled processes at once

Open kavon opened this issue 5 years ago • 0 comments

We run into some system-wide perf_events buffer limitations when many halo-enabled processes are running at once. See the comment here: https://github.com/halo-project/llvm/commit/e389423495329d4dc2840e00ed4ce829e4ab08d0

We had to change the noop test to not spawn 50 clients on the same system because of a pretty low default kernel limit on the number of pages that can be used for perf_events buffers. Now that we have to create one buffer per core, it's quite possible to hit this limit without some care.

One way around this problem would be to only gather sampling data from some CPUs and/or reduce the number of pages in the ring buffer. We can also find out how to increase /proc/sys/kernel/perf_event_mlock_kb and tell users to follow those instructions.

kavon avatar Jan 24 '20 21:01 kavon