bench-rest icon indicating copy to clipboard operation
bench-rest copied to clipboard

Published 20 hours ago • verified publisher icon jeffbski

Question: Ramping requests

Open bapti opened this issue 10 years ago • 3 comments

I was wondering if it's possible to ramp the traffic up over a period of time?

Thanks Neil

bapti avatar Mar 18 '15 12:03 bapti

@bapti I believe it wouldn't be that hard to add, since I would just need to adjust the async queue.concurrency manually over time rather than just kicking off with the max value.

Can you explain your use case and maybe what type of api and/or command options you would prefer to have?

Also would you need any other changes to the output to meet your needs?

jeffbski avatar Mar 18 '15 13:03 jeffbski

@jeffbski that's good to know, I get 1 engineering day a week so I'll probably give it a go within the next week or two.

I've done some stress testing before and if you hit the server too hard to begin with without proper warmup then the server will just fall over.

I'd be interested in hooking it up to a CI server, running nightly and failing if the values fall below levels. I guess I could do this by wrapping the bench-rest run in something like a mocha test. What's your thoughts around this? I think that having a nightly benchmark is quite an exciting thing to add to a project. I'm really keen to try out things like different JSON serialisers and would love to reliably see the differences that making these kind of changes make to the performance of the API.

bapti avatar Mar 18 '15 21:03 bapti

@bapti Excellent, if you get stuck let me know. You can send me a pull request when ready.

Yes, it is an great idea to hook this up to CI server or even git-hook on commit just to catch performance regressions early. I guess you could run it via mocha (may need to set the timeout higher, depending on how long the runs) and then compare that it doesn't vary too much.

However what we ended up doing at a company I worked at was to save some of the benchmarks metrics along with the date, time, and git commit. We would then plot them so you can visually see how things are progressing. We would plot the numbers for several different benchmarks including the overall, that way you can see individual parts as well as the whole over time.

Very powerful. I now recommend setting something up like that from the beginning so you can track progress all along the way.

jeffbski avatar Mar 18 '15 22:03 jeffbski