microsoft
[Feature]: Option to list reporter to not try and update terminal lines
🚀 Feature Request
Instead of updating existing output lines (if output is a tty) when a test resolves, just go ahead and print another status line, as if the output wasn't a tty.
Example
For tests that emit useful output via console.log, when the list reporter is used in a TTY, frequently the last line of a tests console.log output is overwritten when the test resolves. In that case, the reporter could be instructed to just emit another line (as if the output is not a TTY), in a similar fashion to the existing print steps option.
defineConfig({
reporters: [['list', {alwaysAppend: true}]]
})
Motivation
I run tests directly from a TTY, and have tests that output useful/interesting data as they run. In addition, I frequently "debug via printf" when tests are doing unexpected things. Often the list reporter will overwrite the "interesting" log I just added with the "the test is done" update. This is a very small annoyance, but it's also a frequent one, and it's reduction would be nice.
I did try to just extend the existing reporter, but the actual reporter classes aren't exported. Having the default reporters exported publicly would also allow the issue to be solved.
It sounds like you want to use PLAYWRIGHT_FORCE_TTY=0. This is what is used in CI under normal circumstances. Is there some reason this doesn't work for you?
The other TTY decorations (utf8 chars, color) are useful and I'd prefer not to lose them.
You can try setting FORCE_COLOR=1.
Can you provide a repro of the last line of output disappearing using line reporter? It sounds like you might want list reporter, but I also want to make sure we don't have a bug here.
So the specific issue seems to if the line for the initial test status has scrolled out of the buffer, the reporter will update whatever the top most line is here. I've attached a video, and you can repro this with the "default" playwright setup (eg: npm install @playwright/latest) by changing the reporter to line and adding this to the top of the "has title" test that's generated (well and adding the workerIndex to the test method). I could push the reproduction up to a repo if that's useful, but it's not very much change.
for(const i of Array(10).keys()) {
console.log("%s Log %d", workerInfo.workerIndex, i)
}
https://github.com/user-attachments/assets/1a0329c4-17a9-40fe-b9ea-5e99f16f663a
Now that I actually bothered to figure out what's happening, perhaps this is an actual bug and in this scenario the reporter shouldn't try to update the existing line?
I could do a PR to update the reporter to not try to update, and just append, if it's trying to move above the above the height of the TTY? I think that would fix the behavior.
Please provide a full reproduction. I attempted to follow your steps and did not experience the issue.
Please also provide your operating system, Playwright version, and terminal emulator.
https://github.com/jmoses/playwrite-reporter-repro
playwright 1.52.0 mac os 15.5 (24F74) iterm Build 3.5.11 also reproduced in regular terminal
One thing to note is that the "test started" lines have to scroll off the top of the terminal to see the issue. If your terminal is full screen height there's probably not enough test output to trigger it.
Thank you for the repro. This is a clear issue, though I'm unsure what we should do about it due to the complexity of managing TTY state. We are naively assuming the terminal buffer is exactly our screen size (and thus no scrolling), so when a scroll does occur, we clobber other logs.
https://github.com/microsoft/playwright/blob/e6bf21f0f5fcfc84a4acc3b001ce434eaa311302/packages/playwright/src/reporters/list.ts#L233-L240
Yeah, exactly. I think the issue could be at least partially addressed by determining if "update existing line" logic would scroll up past the TTY height, and if so just append instead?