benchexec
benchexec copied to clipboard
sosy-lab
BenchExec: A Framework for Reliable Benchmarking and Resource Measurement
In order to prevent (accidental) fork bombs and exhaustion of the number of available PIDs, BenchExec should use the [pid cgroup controller](https://www.kernel.org/doc/Documentation/cgroup-v1/pids.txt) (if available) and set a limit of concurrent...
Currently, benchexec only accepts B, kB, MB, GB, and TB as units for memory limits. Because GiB is the usually used unit, it would be nice to also support it.
BenchExec should put itself in a cgroup just like it does for the benchmarked process. This will be necessary for cgroup-v2 support anyway (#133), and could be used to measure...
We could add links to the result XML files into the HTML tables, such that we encourage users who publish tables to also publish the raw results, and encourage users...
The systemd unit that creates the necessary cgroups could be bundled in the Debian package such that cgroups setup is done automatically when the package is installed. The package could...
We might want to provide API documentation in a nicely readable way, at least for those parts of BenchExec where we expect others to call the API (at the very...
Currently we create cgroups ourselves, but the kernel is moving in the direction of allowing only a single process to manipulate the cgroup hierarchy. This would be systemd in practice....
BenchExec could get an additional benchmarking mode, where it does not get a benchmark XML file, but a result file, and then re-executes all the runs that are present in...
This year we have added new demo category for SV-COMP'2017 called `c/ldv-multiproperty`. Each task in the category contains several properties to check. The properties are listed in the corresponding property...
Apparently it is possible to measure how much a process utilizes the GPU for computations, there exist [tools for this](http://askubuntu.com/q/387594/51272). BenchExec could probably get support for this as well.