termshark

termshark copied to clipboard

I didn't look very hard so I may have misunderstood what you're actually doing, but https://github.com/gcla/termshark/blob/master/scripts/simple-tests.sh appears to say something about UI tests.

You might be better off using a tool like Tcl's Expect, here's some testing tools from a bit of googling: https://archive.vn/UDeAd ( https://spin.atomicobject.com/2016/01/11/command-line-interface-testing-tools/ ) (also mentions autoexpect, which I haven't used but sounds very interesting!).

Various languages have ports of expect, for example https://github.com/pexpect/pexpect .

There seems to be an introduction to autoexpect here: http://expect.sourceforge.net/doc/autoexpect.pdf

(Somewhat tangentially, asciinema is nice for recordings)

Hi @deliciouslytyped - thanks for the suggestion. I'll write here some disorganized thoughts on testing. The simple-tests.sh script was an attempt to test termshark from the outside; a poor-man's black-box test. There are some unit tests and component tests within termshark, but they're inadequate. As the program has grown, the number of features has grown, and there are now many small details that I implemented deliberately, especially around the UI. The unit tests don't capture any of that, and now I worry, as I change things, that I will inadvertently break something there. I have to look at my notes, go back through git blame, etc... So I think I really need to capture all these behaviors in some sort of black-box test set. That means driving termshark in an automated fashion, in a manner similar to expect. I'd need some way to know when the action taken by the user has had a chance to filter its way through termshark, and the UI has finished reflecting that action (in case it doesn't work). So maybe something simple like a log entry saying termshark is now idle again - no user-processing is pending. I'd also need a way to scrape the UI, to check for the expected results. That could start out simple, like in simple-tests.sh, with a grep. Ideally I'd like to be able to ask if certain elements are at the right coordinates, that colors are correct, if elements are positioned correctly relative to each other. Maybe that would be better done inside termshark with some simple way of querying the widget hierarchy at render time. I also noticed tcell v3 advertises improved testability - https://github.com/gdamore/tcell#testability - which I haven't looked into yet. Presumably that would be at the level of the 2d screen, so not structural. Now I'm rambling, so I'll stop :smile:

I'm going to think about this for v2.5. Maybe it's a separate project.

I'm not sure this is relevant but an interesting thing is that the qtile window manager has quite a thorough command line. I'm curious if in general there are any good UI testing methodologies anywhere and where one can find out more....

The impression I get is that yes, frameworks generally tend to implement their own things that can be hooked into.

I think I saw something that uses Qt-s accessibility features to do something. I recently started working with Angular and that seems to have some sort of end-to-end test system that I haven't looked into yet.

Hi @deliciouslytyped - thanks for the suggestions I'll follow those up. I haven't used qtile before. Yesterday I found an old bare-bones framework built as an extension of expect, called term_expect: https://github.com/dineshkummarc/expect-src/blob/master/example/term_expect. That's the closest match so far to the way I'd imagined this working.

I wondered about building a testing tool on top of the gowid terminal widget because then it might be useful as a more modern way to test any terminal application, not just those based on tcell + Go. The testing application could then be the terminal widget with some programmed extensions that capture the expect-style rules you want to apply (written in Go).

I'm wary though because I might just end up building something that is like expect except less well-thought out... And while the terminal widget does pretty well against vttest, I don't know how well it stands up to modern TUI applications - though it does render termshark acceptably.