k6 icon indicating copy to clipboard operation
k6 copied to clipboard

Ability to display intermediate result while the test is running

Open sniku opened this issue 5 years ago • 6 comments

This is a suggestion for the k6 process to listen to user input and react to commands while the test is running.

Feature Description

Specifically, I would like to be able to press R and to get intermediate results while my test is running. This is especially useful for long-running soak tests, that can take several hours to complete.

Currently, it's possible to get intermediate results by running a k6 stats command in another terminal window. This is however not ideal because the format is not human readable.

I suggest the intermediate result to have the same format as the regular end-of-test output k6 normally generates.

sniku avatar Jan 22 '20 12:01 sniku

Now that we have https://github.com/loadimpact/k6/pull/1768, this issue should be relatively easy to implement. Though we might need to refactor the js.Runner a bit, if we want to be a bit more efficient and not spawn a VU every time we call handleSummary(). On the other hand, that would require a bit more locking so that if the user presses R multiple times in a rapid succession, we don't try to run handleSummary() more than once concurrently in the same runtime... :man_shrugging:

Also, a keyboard letter like R might not be the best UI for this, ~a signal like Ctrl+Z (SIGTSTP) might be better~? Or maybe not, considering its current use to move the process to the background... :disappointed:

na-- avatar Jan 21 '21 12:01 na--

As has become apparent in the course of https://github.com/loadimpact/k6/pull/1849, this feature has a lot of hidden complexity and deserves a lot of consideration before we implement it. Specifically, we either have to accept a poor UX, or we have to deal with a lot of terminal strangeness... :disappointed:

na-- avatar Feb 17 '21 15:02 na--

I don't see why we can't compromise and allow this to work with Enter instead of R. It avoids the mess of switching terminal modes and from my tests on top of a previous iteration of #1849 works quite well. Enter seems like a sensible binding for this type of output, and we won't have to implement some complex key listening functionality, which we arguably don't need.

imiric avatar Feb 17 '21 15:02 imiric

My requirement of listening to R input was quite arbitrary. If listening to Enter removes complexities from this task, then I think we should change the requirement.

sniku avatar Feb 19 '21 14:02 sniku

Then doesn't it become too easy enough to trigger this accidentally? And very important that you don't accidentally pipe something to k6's stdin... Binding Enter would also preclude us from doing pretty much anything else with user interaction, i.e. we'd have to make a breaking change if we ever want more similar features in k6.

Mind you, I am not against this, I just want us to spend a bit more time thinking about it before we try again... :sweat_smile: It's not a very important feature, so I think we can let this lie for a while before we try again.

na-- avatar Feb 19 '21 15:02 na--

There was an attempt to implement this issue (https://github.com/grafana/k6/pull/1849), however, as it's obvious by the comments in the PR, there were some unexpectedly tricky issues around it (mostly in how the mid-test handleSummary() is invoked). I'd go as far as saying that listening for a key stroke is probably a very bad way to go about this issue, maybe exposing a REST API call or listening to signals would work better if we ever want to implement this in the future. For now, I'll keep this issue open, but it definitely still seems like a very low prio one.

na-- avatar Mar 31 '22 09:03 na--