Stefan Marr

Sounds to me as if interop is slowly becoming its own language.

@timfel this wasn't the original vision as I understand it from @grimmerm's work. Originally, it was quite minimal in scope, a behavioral interface. It was not a full featured language....

One of the issues I am not yet quite sure about is that we already have a notion of `machine` or rather `machines`. Introduced with https://github.com/smarr/ReBench/pull/161 and in the schema...

It looks like this is indeed an issue in the ReBenchDB integration tests: https://github.com/smarr/ReBenchDB/actions/runs/11124536077/job/30910272896#step:11:54 ``` Minimizing noise with rebench-denoise failed possibly causing benchmark results to vary more. Error: Traceback (most...

I am not too concerned about security really. We throw security out of the window with denoise in many ways already. So, it's mostly about reproducibility. And yes, a default...

Thanks for the interest in ReBench, unfortunately, I don't have the resources to support Windows. If you or others are interested in making it work on Windows, that's great, and...

Yeah, the luasocket module does a very similar thing: https://github.com/lunarmodules/luasocket/blob/master/src/timeout.c#L134-L139 Not sure what the most user-friendly approach here would be. I suppose the harness could warn if no reliable clock/time...

Hm, hard errors are problematic for testing and maintaining the benchmarks. But I suppose one could setup CI so that it has the recommended clock. PRs welcome...

At the moment, we request explicitly the use of a shell here: https://github.com/smarr/ReBench/blob/master/rebench/executor.py#L532 However, we run build commands without shell here: https://github.com/smarr/ReBench/blob/master/rebench/executor.py#L397

Yeah, I fear changing that is a major change to the benchmark. There are a couple of other rather specific choices in the original code I got from SOM: https://github.com/SOM-st/som-java/commit/3a69207b7ae5672b7ef4986de649dd0da308f146#diff-5241273eecb6c5e75da48d83df5ec1de46647780885655761b830e888ec9d486...