bubbletea feat: teatest package

feat: teatest package

Open caarlos0 opened this issue 2 years ago • 10 comments

Introducing the teatest package - a very small package with testing helpers for bubbletea apps.

You basically test a model, giving the model itself, a set of interactions (tea.Msg's) and an assertion, which is basically asserting the output against a given golden file.

Its a very raw idea, any feedback welcome.

Jun 21 '22 12:06 caarlos0

Is this PR a candidate for merging anytime soon? I have been using bubbletea for a little while now, it is a fantastic library, but it does mean you end up with a program that is largely untested through automation. I have been thinking of writing my own library for testing bubbletea, but noticed this PR.

Jul 12 '22 12:07 purpleclay

@purpleclay we're trying it out in some places to see how it feels like... if you wanna give it a try and let us know what you think, it would be greatly appreciated as well (and you can help driving how it should look like).

You can play with it by getting this branch instead:

go get github.com/charmbracelet/bubbletea@test

Jul 12 '22 19:07 caarlos0

@caarlos0 sure I will try it out on my current project. I guess the only question I would have while trying to use the current flavour of teatest is how to generate the golden image files that are used for comparison. Especially if you have a complex TUI. It feels like validating the entire output is the safest option

Jul 12 '22 19:07 purpleclay

@purpleclay you can run go test ./yourpackage/... -update and it will generate them for you

Jul 12 '22 19:07 caarlos0

Hi @caarlos0 I have attempted to use teatest by just writing a single test case. You can view my test here:

https://github.com/purpleclay/dns53/blob/teatest/internal/tui/dashboard_test.go

So here are my observations (btw I am liking the fact I can test my TUI):

My Update() method is driven by the initial Init() and needs it to return a tea.Msg containing EC2 metadata. This requires me to add an enforced sleep. Is this something most people will have to do? As I noticed it in your simple example that you do this also. Is there not a way for teatest to wait for all Init commands to complete automatically?
The generation of golden files through the -update flag is a nice touch. Would it feel more intuitive to do this through a go:generate?
Do you envisage people testing partial parts of the terminal output within teatest.TestModel(func(out []byte) {})? I imagine I would always use the golden file approach. Attempting to assert against fragments of text would require me to include special characters e.g. [1;;mPHZ:[0m AAAAAAAAAAAAAAAAAAAAAAAA [testing] . Partial matching would be useful if these characters could be stripped from the output
It seems you cannot run any of the teatest test cases if you provide the -race flag to go test. Is that by design? This is the current output from my test:
I am thinking of including a ticker in my TUI. I am wondering what impact this would have on the golden file generated. Would we get sporadic failures

data race details

==================
WARNING: DATA RACE
Write at 0x00c0003b7eb0 by goroutine 8:
  bytes.(*Buffer).Write()
      /usr/local/Cellar/go/1.18.3/libexec/src/bytes/buffer.go:169 +0x44
  fmt.Fprintf()
      /usr/local/Cellar/go/1.18.3/libexec/src/fmt/print.go:205 +0xb1
  github.com/charmbracelet/bubbletea.clearLine()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/screen.go:19 +0x85
  github.com/charmbracelet/bubbletea.(*standardRenderer).stop()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/standard_renderer.go:75 +0x31
  github.com/charmbracelet/bubbletea.(*Program).shutdown()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:588 +0xd1
  github.com/charmbracelet/bubbletea.(*Program).StartReturningModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:496 +0x1cd1
  github.com/charmbracelet/bubbletea.(*Program).Start()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:549 +0x52
  github.com/charmbracelet/bubbletea/teatest.TestModel.func1()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:35 +0x53

Previous write at 0x00c0003b7eb0 by goroutine 7:
  bytes.(*Buffer).Write()
      /usr/local/Cellar/go/1.18.3/libexec/src/bytes/buffer.go:169 +0x44
  fmt.Fprintf()
      /usr/local/Cellar/go/1.18.3/libexec/src/fmt/print.go:205 +0xb1
  github.com/charmbracelet/bubbletea.showCursor()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/screen.go:15 +0x4d
  github.com/charmbracelet/bubbletea.Program.restoreTerminalState()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tty.go:30 +0x25
  github.com/charmbracelet/bubbletea.(*Program).ReleaseTerminal()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:687 +0x164
  github.com/charmbracelet/bubbletea/teatest.TestModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:44 +0x378
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:50 +0x1dc
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:42 +0x19b
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x47

Goroutine 8 (running) created at:
  github.com/charmbracelet/bubbletea/teatest.TestModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:34 +0x2fd
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:50 +0x1dc
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:42 +0x19b
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x47

Goroutine 7 (running) created at:
  testing.(*T).Run()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x724
  testing.runTests.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1839 +0x99
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.runTests()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1837 +0x7e4
  testing.(*M).Run()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1719 +0xa71
  main.main()
      _testmain.go:99 +0x3a9
==================
==================
WARNING: DATA RACE
Read at 0x00c0003b7e90 by goroutine 8:
  bytes.(*Buffer).tryGrowByReslice()
      /usr/local/Cellar/go/1.18.3/libexec/src/bytes/buffer.go:107 +0x52
  bytes.(*Buffer).Write()
      /usr/local/Cellar/go/1.18.3/libexec/src/bytes/buffer.go:170 +0x18
  fmt.Fprintf()
      /usr/local/Cellar/go/1.18.3/libexec/src/fmt/print.go:205 +0xb1
  github.com/charmbracelet/bubbletea.clearLine()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/screen.go:19 +0x85
  github.com/charmbracelet/bubbletea.(*standardRenderer).stop()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/standard_renderer.go:75 +0x31
  github.com/charmbracelet/bubbletea.(*Program).shutdown()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:588 +0xd1
  github.com/charmbracelet/bubbletea.(*Program).StartReturningModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:496 +0x1cd1
  github.com/charmbracelet/bubbletea.(*Program).Start()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:549 +0x52
  github.com/charmbracelet/bubbletea/teatest.TestModel.func1()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:35 +0x53

Previous write at 0x00c0003b7e90 by goroutine 7:
  bytes.(*Buffer).tryGrowByReslice()
      /usr/local/Cellar/go/1.18.3/libexec/src/bytes/buffer.go:108 +0xb3
  bytes.(*Buffer).Write()
      /usr/local/Cellar/go/1.18.3/libexec/src/bytes/buffer.go:170 +0x18
  fmt.Fprintf()
      /usr/local/Cellar/go/1.18.3/libexec/src/fmt/print.go:205 +0xb1
  github.com/charmbracelet/bubbletea.showCursor()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/screen.go:15 +0x4d
  github.com/charmbracelet/bubbletea.Program.restoreTerminalState()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tty.go:30 +0x25
  github.com/charmbracelet/bubbletea.(*Program).ReleaseTerminal()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:687 +0x164
  github.com/charmbracelet/bubbletea/teatest.TestModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:44 +0x378
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:50 +0x1dc
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:42 +0x19b
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x47

Goroutine 8 (running) created at:
  github.com/charmbracelet/bubbletea/teatest.TestModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:34 +0x2fd
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:50 +0x1dc
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:42 +0x19b
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x47

Goroutine 7 (running) created at:
  testing.(*T).Run()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x724
  testing.runTests.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1839 +0x99
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.runTests()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1837 +0x7e4
  testing.(*M).Run()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1719 +0xa71
  main.main()
      _testmain.go:99 +0x3a9
==================
==================
WARNING: DATA RACE
Read at 0x00c000156ce8 by goroutine 8:
  runtime.racereadrange()
      <autogenerated>:1 +0x1b
  github.com/charmbracelet/bubbletea.(*Program).StartReturningModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:496 +0x1cd1
  github.com/charmbracelet/bubbletea.(*Program).Start()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:549 +0x52
  github.com/charmbracelet/bubbletea/teatest.TestModel.func1()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:35 +0x53

Previous write at 0x00c000156ce9 by goroutine 7:
  github.com/charmbracelet/bubbletea.(*Program).ReleaseTerminal()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/tea.go:682 +0xc4
  github.com/charmbracelet/bubbletea/teatest.TestModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:44 +0x378
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:50 +0x1dc
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:42 +0x19b
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x47

Goroutine 8 (running) created at:
  github.com/charmbracelet/bubbletea/teatest.TestModel()
      /Users/purpleclay/go/pkg/mod/github.com/charmbracelet/[email protected]/teatest/teatest.go:34 +0x2fd
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:50 +0x1dc
  github.com/purpleclay/dns53/internal/tui_test.TestDashboard()
      /Users/purpleclay/dev/github.com/purpleclay/dns53/internal/tui/dashboard_test.go:42 +0x19b
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x47

Goroutine 7 (running) created at:
  testing.(*T).Run()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1486 +0x724
  testing.runTests.func1()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1839 +0x99
  testing.tRunner()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1439 +0x213
  testing.runTests()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1837 +0x7e4
  testing.(*M).Run()
      /usr/local/Cellar/go/1.18.3/libexec/src/testing/testing.go:1719 +0xa71
  main.main()
      _testmain.go:99 +0x3a9
==================
--- FAIL: TestDashboard (0.21s)
    testing.go:1312: race detected during execution of test

Jul 18 '22 05:07 purpleclay

ahh, glad you got to test it @purpleclay!

My Update() method is driven by the initial Init() and needs it to return a tea.Msg containing EC2 metadata. This requires me to add an enforced sleep. Is this something most people will have to do? As I noticed it in your simple example that you do this also. Is there not a way for teatest to wait for all Init commands to complete automatically?

in that example case, the sleep is to ensure some time passed so I can properly test the output... that said, since commands run in another goroutine, it might be needed to either sleep or sync with a chan or something like that...

The generation of golden files through the -update flag is a nice touch. Would it feel more intuitive to do this through a go:generate?

I think you can put a //go:generate go test -update where you need...

Do you envisage people testing partial parts of the terminal output within teatest.TestModel(func(out []byte) {})? I imagine I would always use the golden file approach. Attempting to assert against fragments of text would require me to include special characters e.g. �[1;;mPHZ:�[0m AAAAAAAAAAAAAAAAAAAAAAAA [testing]. Partial matching would be useful if these characters could be stripped from the output

I think some people might, for simpler outputs...

It seems you cannot run any of the teatest test cases if you provide the -race flag to go test. Is that by design? This is the current output from my test:

investigating...

I am thinking of including a ticker in my TUI. I am wondering what impact this would have on the golden file generated. Would we get sporadic failures

depending on your project, yes... unfortunately.. I don't have a good solution for that... but I'm open to hear any ideas you might have...

Jul 21 '22 12:07 caarlos0

@purpleclay the race should be fixed now!

Jul 21 '22 13:07 caarlos0

@purpleclay the race should be fixed now!

Awesome. I will give it another try

Jul 21 '22 15:07 purpleclay

I think it's possible, yes. Will work on this later 🤘

Sep 16 '22 13:09 caarlos0

@caarlos0 is there a potential timeframe of when this teatest package will make it into the main branch?

Sep 16 '22 14:09 purpleclay

@caarlos0 is there a potential timeframe of when this teatest package will make it into the main branch?

no specific timelines yet, no :(

Sep 17 '22 01:09 caarlos0

On the test branch, using ./examples/simple when I run go test -v, for all 3 of the instances I'm getting: (where X = 1, 2, 3)

=== CONT  TestApp/X
    main_test.go:30: output does not match, diff:
        
        1,2c1,2
        Hi. This program will exit in 9 seconds. To quit sooner press any key.
        ---
        Hi. This program will exit in 9 seconds. To quit sooner press any key.

Not sure what is causing the example to fail for me.

After running: go test -update (which passes) and running go test -v again I get:

=== CONT  TestApp/0
    main_test.go:30: output does not match, diff:
        
        3c3

        \ No newline at end of file
        ---

        \ No newline at end of file
=== CONT  TestApp/2
    main_test.go:30: output does not match, diff:
        
        3c3

        \ No newline at end of file
        ---

        \ No newline at end of file

Oct 15 '22 06:10 erwin

What do you think of adding a RequireRegexpOutput to teatest?

        teatest.WithRequiredOutputChecker(func(out []byte) {
          teatest.RequireRegexpOutput(t, out, `Hi\. This program will exit in \d seconds\. To quit sooner press any key\.`)
        }),

Then inside of teatest.go just add something like:

func RequireRegexpOutput(tb testing.TB, out[]byte, re string) {
  rexp, err := regexp.Compile(re)
  if err != nil {
    tb.Fatalf("Error Compiling Regexp: %s", err)
  }
  match := rexp.Match(out)
  if ! match {
    tb.Fatalf("output does not match:\nRegExp: %s\nOutput: %s", rexp.String(), string(out))
  }
}

Over time could improve that to help people narrow it down to exactly the portion of the RegExp that caused the match to fail. I'll have to do some research on that.

Oct 16 '22 05:10 erwin

I noticed when using teatest.withProgramInstructions() and sending tea.WindowSizeMsg, the test will hang indefinitely.

   p.Send("ignored msg")
>> p.Send(tea.WindowSizeMsg{ Height: 24 })
   p.Send(tea.KeyMsg{ Type: tea.KeyEnter, })

Would it be feasible to do something like mock the bubbletea/signals_unix -> listenForResize stuff in teatest so that terminal windows would have a defined size?

Personally, I would like to test tea components at various sizes and know that the planned information appeared on screen at each size.

Oct 17 '22 01:10 erwin

and sending tea.WindowSizeMsg, the test will hang indefinitely.

I think you need to send both width and height for it to work...

Oct 17 '22 16:10 caarlos0

Personally, I would like to test tea components at various sizes and know that the planned information appeared on screen at each size.

fwiw added a WithInitialTermSize option

Oct 17 '22 18:10 caarlos0

Related work: https://github.com/charmbracelet/bubbletea/pull/536

This provides a generic event filter that we can use to inspect events in tests. It also exports QuitMsg so you can identify such messages.

Oct 18 '22 04:10 muesli

That looks awesome @muesli !

Oct 18 '22 12:10 caarlos0

In examples/simple/main_test.go on the test branch, if the model is initialized to count down for 10 seconds, and then sleeps for 1.2 seconds on line 19, then shouldn't we be testing to make sure the value of the model has been decremented from 10 to 9 on line 31?

My apologies if I'm wrong and just misunderstanding the intention here.

If we should be testing for 9 rather than 10 here, then in teatest -> TestModel dont' we need to get the Update()ed model back after tea.Run before running opts.validateModel?

If we do want to get the models final status returned by bubbletea's Run() funnction, so teatest.go func TestModel, perhaps change to:

func TestModel(tb testing.TB, m tea.Model, options ...TestOption) {
	var in bytes.Buffer
	var out bytes.Buffer
>>>	var err error

Then just assign the model m to the output of p.Run (currently being ignored)

	go func() {
>>>		if m, err = p.Run(); err != nil {
			tb.Fatalf("app failed: %s", err)
		}
		done <- true
	}()

And update the example examples/simple/main_test.go to check the model m == 9.

		teatest.WithValidateFinalModel(func(mm tea.Model) error {
			m := mm.(model)
			if m != 9 {
				return fmt.Errorf("expected model to be 10, was %d", m)
			}
			return nil
		}),

Oct 20 '22 16:10 erwin

ah, indeed, good catch @erwin !

fixed on last commit

Oct 21 '22 12:10 caarlos0

I was writing some tests today and noticed cases where I should make sure that a RegEx does not match. For this particular example, I have an array assigned to the viewport bubble, and based on the size and keyboard input, some items should be present, and others not present.

Your teatest.WithRequiredRegexpOutput works great to test for output you want present, but if the match fails, it will call t.Fatal so you can't easily invert it to test for !WithRequiredRegexpOutput

@caarlos0 What do you think of another helper in teatest for cases where you don't want the regexp to match?

Oct 25 '22 05:10 erwin

@erwin tbh I'm thinking about removing the regex one completely... you already have the testing.T and the output, so it just does the regex matching... which many libs (like testify) already provide helpers for...

IMHO teatest should help setting up the app for testing, but that does not include implemente a test matchers library 🤔

Oct 25 '22 12:10 caarlos0

I thought many convenience functions like regex wrappers (and any other widely useful ones) may make sense as part of teatest, because you already have what I perceived as the rather unusual (very special case?) RequireEqualOutput function, and something like string matching seems more generally useful.

I think functions that auto strip escape sequences from out also come to mind as useful and specific to TUI development.

The Regex helper is very trivial code, and as you point out, out is exposed, so people can easily implement whatever they want. I don't have any deep feeling about whether or not that function makes it into teatest. Since not everyone loves Regex's, perhaps direct string matching stuff would be even more generally useful, and make for more popular examples.

I suppose my point is that the testing library and the corresponding example code both SHOUT LOUDLY to new developers as they first adopt bubbletea. Every developer that uses bubbletea will experiment with at least some of the examples. The smoother and cooler that experience, the more bubbletea and charm.sh can grow.

I think that the RequireEqualOutput function comes across as too strange, and maybe just needs a comment to sell why it's such a great way to test. Or if you think everybody should be using testify in their tests, maybe some more in the examples that show why testify is so valuable.

So far, teatest is very useful to me and I am very glad that you've written it!

Oct 28 '22 09:10 erwin

@erwin thanks for the feedback!

RequireEqualOutput does a few more things, more related to golden file testing: create/update the .golden files, compare with diff, etc... maybe that could/should be in another repository?

Nov 01 '22 02:11 caarlos0

maybe that could/should be in another repository?

That's an interesting idea! Or maybe just another package name, depending on how you anticipate it growing!

Nov 01 '22 13:11 erwin

I come from a webdev/React background and we typically use React Testing Library to test our UIs, so naturally, I was looking for a similar approach to test my TUI apps. I've been experimenting with this small package built on top of termtest to achieve this. The idea is pretty similar to this approach, but instead of sending a sequence of commands and then doing an assert at the end, the flow is more like

loadApp()
send("some input")
assert(app.output.contains("some output"))

send("some more input")
assert(app.output.contains("some more output"))

For more complex apps, I tend to prefer fine-grained asserts rather than matching the entire output so that all your tests don't fail as soon as you change one small part of your UI. Snapshot testing in React has somewhat fallen out of favor for this reason.

My app also heavily relies on styling to show UI changes so being able to do asserts on the foreground/background colors after each interaction is quite important - something like output(row, col).background == green.

Not saying this approach is better or worse, just some other ideas to consider.

Nov 01 '22 14:11 aschey

@aschey ... that's an interesting concept... will experiment with it a bit

Nov 01 '22 16:11 caarlos0

okay, I think we can have both options (current and the new proposed), will push an example in a few

Nov 01 '22 16:11 caarlos0

okay, pushed, lemme know what you think!

I figured the in in the WithProgramInteractions was not really being used, so I changed that. Let me know if you think that assumption is wrong.

Nov 01 '22 16:11 caarlos0

Nice, looks good! Since UI updates happen asynchronously, most UI testing libraries have a default timeout for any assertions so you don't have to hardcode sleeps in your program. Something like

err := p.WaitFor(func (out []byte) bool { 
   return bytes.Contains(out, []byte("This program will exit in 9 seconds"))
}, 2 * time.Second)

which will wait for up to 2 seconds before returning an error if the condition fails might be useful. Of course users could write this function themselves, but I think it would be so commonly used that having it baked in might be beneficial.

Nov 01 '22 17:11 aschey

bubbletea bubbletea copied to clipboard

feat: teatest package

bubbletea
bubbletea copied to clipboard