go-internal icon indicating copy to clipboard operation
go-internal copied to clipboard

Allow custom commands to write to stdout and stderr

Open matthewmueller opened this issue 3 years ago • 1 comments

Hey there, I've had a couple cases where it'd be nice to be able to write to stdout and stderr from inside a custom command.

For example:

func Test(t *testing.T) {
	p := testscript.Params{
		Cmds: map[string]func(ts *testscript.TestScript, neg bool, args []string){
			"render": render,
		},
	}
	testscript.Run(t, p)
}

func render(ts *testscript.TestScript, neg bool, args []string) {
    html, err := render.Render(ts.MkAbs(args))
    ts.Check(err)
    _ = html
}
render 'index.gohtml'

-- index.gohtml --
<html>
  <body>
    <h1>{{.}}</h1>
  </body>
</html>

Unfortunately I haven't found a way to use the command's results within the test. The workaround I use now is to write a file and then read from it.

What I'd like to be able to do is write a testscript like this:

render 'index.gohtml'
stdout '<h1>'

-- index.gohtml --
<html>
  <body>
    <h1>{{.}}</h1>
  </body>
</html>

And the failure case:

! render 'index.gohtml'
stderr 'bad HTML'

-- index.gohtml --
<html

Thanks for extracting and improving this package!

matthewmueller avatar May 02 '21 12:05 matthewmueller

FWIW opened a PR for this one here: https://github.com/rogpeppe/go-internal/pull/140

matthewmueller avatar May 02 '21 13:05 matthewmueller

Picking up API conversation from https://github.com/rogpeppe/go-internal/pull/217#pullrequestreview-1401240139

  • Please add a test case for a command that writes to stdout and stderr, and then returns an error. I wonder if stderr is then kept at all, for example.

I don't think this is actually something we can test. A Params.Cmds command has complete control on whether execution is a failure or not. The neg parameter indicates whether the user intended the command to fail or not. So the only "error" situation is when the execution of such a command does not concur with neg. In that situation, execution terminates immediately.

The real problem is the API here. We're trying to work around the fact that a Params.Cmds command should return stdout and stderr, rather than setting it via the approach proposed in #217, an approach which implies it can be done multiple times... which isn't the case. Just as such a command can only call (*TestScript).Fatalf() once.

Fixing this properly, however, would either require us to declare a new API via another field on Params, or by loosening the type of Params.Cmds to allow for functions that have the signature:

func(ts *TestScript, neg bool, args []string) (stdout, stderr string, err error)

myitcv avatar Apr 26 '23 07:04 myitcv