usebruno
Improve Runner to run load tests
Right now, the Runner module allows us to run all the requests in a given collection and see a nice results page updated in real time.
In my opinion this is already an excellent starting point, and very close to what other apps (e.g: Postman) offer.
However, I think by adding 3 very specific features, it would easily match and surpass those offerings, making it an excellent load testing tool:
-
Allow specifying a number of iterations for the collection Right now we can only run the whole collection once. I may want to create a folder/collection for a specific load test, where I want to run just 2 different requests, but 10 times each. This is extremely useful to test rate-limiting algorithms, load balancing systems, etc., and is supported by Postman and others.
-
Allow us to choose whether we want to run requests sequentially or concurrently Going back to the load testing/rate limiting tests, sometimes you just want to launch the same request 100 times, as fast as possible, even if the requests themselves take some time to respond. Currently, Bruno runs all requests in a collection sequentially, with no option for concurrent runs. It always waits for a response before launching the next request. This is NOT supported by Postman, but can be done in Insomnia using plugins.
-
Results pane: include the time taken by each request, perhaps apply different colors depending on result code, and make report exportable. This is again very useful for load testing. You get to see the failed requests at a glance (200 = green, 500 = red, others = orange?), and the timing data clearly shows which requests introduced the most delay. It makes for a nice snapshot that you can easily share with others, either live during videocalls, or by exporting for offline sharing. I guess CSV/HTML/PDF/Markdown would be the usual suspects. Although Postman does allow exporting the results, it only supports JSON format, and the result presentation within the app is quite cluttered and not clean like Bruno's results are.
As a reference, and because pictures speak louder... yaddah yaddah, here's the results table that I created while contributing to an Insomnia plugin (right before they changed their business model and screwed everything up):
Of course this kind of table design probably wouldn't fit with Bruno's style, but just so you get the idea.
Do you think this would be feasible/interesting?
+1 on the option to run all the requests in a collection or folder concurrently. This would be very useful for stress testing/rate limit testing/race condition testing.
I think adding load testing to an API test runner is feature bloat. I can imagine it also requiring quite different performance characteristics of the underlying req/res machinery.
A fruitful direction could be to wrap a bru CLI command inside a script that both runs the command X times in Y seconds and gathers, parses and reports on the results? That would also make that script somewhat independent of the actual HTTP sending code, making it more portable.
"Allow specifying a number of iterations for the collection
Right now we can only run the whole collection once. I may want to create a folder/collection for a specific load test, where I want to run just 2 different requests, but 10 times each. This is extremely useful to test rate-limiting algorithms, load balancing systems, etc., and is supported by Postman and others."
This can be done today by selecting the folder menu and execute run. vs runner from top menu. IMO there is a bug in this menu because it does not permit you to set delay like from top menu but you can work around that in the script and you can set iterations here
