test-infra icon indicating copy to clipboard operation
test-infra copied to clipboard

Improve local running expierience.

Open bwplotka opened this issue 5 years ago • 3 comments

So I am doing lots of microbenchmarks and unfortunately still frontbench is way behind the bash script I developed through the year ;p

At this point I am curious if it's worth adjusting funcbench or actually it's time to start separate tool.. ): What do you think @geekodour ?

My scripts are available here: # Shared @ https://gist.github.com/bwplotka/3b853c31ed11e77c975b9df45d105d74 I feel those might be useful for some people (ideally if done in Go)

bwplotka avatar Aug 25 '20 11:08 bwplotka

Thanks for sharing the scripts :)

I think what you're doing with your script is totally possible to do with funcbench. What I think could improve the local experience would be:

  • nice way to get it installed
  • run benchmarks in an isolated directory that does not change whatever user is working on by any means
  • improve worktree usage
  • have perflock for local. This I think will have to be done via an alias, which we can include in readme for now.

Any other changes you suggest?

geekodour avatar Aug 26 '20 08:08 geekodour

We might want to also enhance storing of old results in some way, like I did in the script. The key flow is that you usually we do changes to commit A and bench, then we improve and check if it's better or worse on commit B, so we compare A, B.

Then we do C, D, E and we usually compare with A and don't want to redo A all the time. Funbench however by design always do A and Target commit

bwplotka avatar Aug 26 '20 08:08 bwplotka

Ah! I see, this will be a nice addition! I'll try to implement this in the coming week then :)

I can't think of any ways this can be used when running funcbench from a github PR so will add this local only for now and keep the github PR flow the same.

geekodour avatar Aug 29 '20 07:08 geekodour