test-infra
test-infra copied to clipboard
Improve local running expierience.
So I am doing lots of microbenchmarks and unfortunately still frontbench is way behind the bash script I developed through the year ;p
At this point I am curious if it's worth adjusting funcbench or actually it's time to start separate tool.. ): What do you think @geekodour ?
My scripts are available here: # Shared @ https://gist.github.com/bwplotka/3b853c31ed11e77c975b9df45d105d74 I feel those might be useful for some people (ideally if done in Go)
Thanks for sharing the scripts :)
I think what you're doing with your script is totally possible to do with funcbench. What I think could improve the local experience would be:
- nice way to get it installed
- run benchmarks in an isolated directory that does not change whatever user is working on by any means
- improve worktree usage
- have perflock for local. This I think will have to be done via an alias, which we can include in readme for now.
Any other changes you suggest?
We might want to also enhance storing of old results in some way, like I did in the script. The key flow is that you usually we do changes to commit A and bench, then we improve and check if it's better or worse on commit B, so we compare A, B.
Then we do C, D, E and we usually compare with A and don't want to redo A all the time. Funbench however by design always do A and Target commit
Ah! I see, this will be a nice addition! I'll try to implement this in the coming week then :)
I can't think of any ways this can be used when running funcbench from a github PR so will add this local only for now and keep the github PR flow the same.