Have a way to show test(s) being run, running individual files, specifying parallel jobs

[osa1]

I have a looping test and goldentests seems to have no way of showing me which test is being run, so I can't find which test it is.

indicatif either shows a progress bar (not helpful to see which tests are running) or nothing (when piped to a file).

goldentests cannot be built without indicatif without changes. When changes are made to build the executable without the indicatif feature it doesn't show the progress at all.

goldentests also can't run only a single file, it always wants a directory and runs everything in it. So I can't do something simple like:

for f in tests/*; do
    echo $f
    goldentests ... $f
done

to find out which test file is getting stuck.

I know this is a lot of feature requests, but it would be great to have:

• A progress indicator that shows files being run.
• Ability to run given list of test files, instead of everything in a directory.
• Specifying parallel jobs.

For the problem with finding a looping test any of these would do the job, but they're also useful on their own. Many other build systems and test runners have flags for these.

[jfecher]

Of these specifying parallel jobs would probably be the simplest.

I suppose running with a given list of files wouldn't be too difficult either if there's a CLI option to accept them but this may be unwieldy for larger file sets without an external script to provide them. I'd be interested in the sort of things you'd want to pass files manually to it would be for since I'd also like to improve the file detection & filtering of it (e.g. only files matching a given sufix like *.an). Supplying things manually as a fallback is useful, I'm just curious as to your specific usecase to see if there is anything else we could better support by default instead of requiring users create external runner scripts.

A progress indicator that shows files being run.

I'm assuming you know about the existing progress bar (which must be manually enabled since if you're just running through cargo t it is hidden). Printing items on separate lines while going back up to update the progress bar line can be finicky, so maybe we'd want to print them to the left/right of the progress bar on the same line (but maybe limit them to ~3 filenames at a time) e.g:

$ goldentests ... args ...
[==============================                                                     ]  foo.ex, bar.ex, baz.ex...

Versus

$ goldentests ... args ...
[=====================================                                                                            ]
Testing foo.ex...
Testing bar.ex...
Testing baz.ex...

[osa1]

Another use case for the --jobs parameter: I modified my interpreter to generate some debug information to a file, but with multiple jobs the interpreter processes modify the file at the same time. If I could do --jobs 1 that would help a lot.

Right now the easiest solution is reinstalling golden-tests without the parallel feature, and then reinstalling it with the feature after debugging.

[osa1]

I'd be interested in the sort of things you'd want to pass files manually to it would be for since I'd also like to improve the file detection & filtering of it (e.g. only files matching a given sufix like *.an).

My use case is just that I want to be able to have extra files (mainly READMEs) in test directories. Right now I can't because goldentests tries to test the file.

[jfecher]

Relevant: https://github.com/jfecher/golden-tests/issues/14

It sounds like filtering for file suffix would be the main solution here.