Basic reporter but with real time progress

[acelaya]

Clear and concise description of the problem

We have been lately trying to get vitest to report the progress of tests in real time, as it does with its default reporter, but skipping the summary of every test file.

Using reporters: ['basic'] does something in those lines, but you don't get any sense of progress until the summary is dumped all at once at the end.

As a workaround, we have defined our own test reporter which extends from DefaultReporter, but overrides onTestModuleEnd as a no-op:

import { DefaultReporter } from 'vitest/reporters';

/**
 * This reporter provides a real-time progress summary during test execution and
 * reports failures in detail but avoids printing a list of successful tests.
 */
export class CustomReporter extends DefaultReporter {
  onTestModuleEnd() {
    // Override this method to suppress individual file results
  }
}

This produces the type of output we are looking for:

https://github.com/user-attachments/assets/872fbf69-c484-4a45-8b21-4a7776e20e80

Versus the output when using reporters: ['basic']:

https://github.com/user-attachments/assets/22a2376f-0203-4daf-a534-de7072ab4715

Not entirely related, but using the basic reporter prints a warning that using ['default', { summary: false }] is equivalent, but it doesn't seem to be 🤔

Suggested solution

Perhaps adding a progress?: boolean option to the basic reporter that ensures there's some sense of progress, would be a good approach.

reporters: [['basic', { progress: true }]

Alternative

I described above we are currently working around this using our custom reporter. It's pretty simple, but I thought others might want something similar.

Additional context

No response

Validations

• [x] Follow our Code of Conduct
• [x] Read the Contributing Guidelines.
• [x] Read the docs.
• [x] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[AriPerkkio]

I'm not really sure I understand what this issue is about. Could you show what's the output that you are seeing, and what you would like to see instead?

[acelaya]

I'm not really sure I understand what this issue is about. Could you show what's the output that you are seeing, and what you would like to see instead?

I added a couple screen recordings above.

The first one is what we would like to see, which is what we get by using our custom reporter. The second one is what we get with reporter: ['basic'].

They are the same once the tests have finished, but during the execution, the default reporter displays the tests that are being executed in real time, giving a sense of progress.

The basic reporter, on the other hand, does not print anything until all tests have been executed, so it feels like it's stuck until it finishes.

[AriPerkkio]

The basic reporter is being removed, as it's equal to reporter: [ ["default", { summary: false }] ].

What's the difference in output when you compare default reporter and your custom one?

[acelaya]

This is what we see with reporter: ['default']:

https://github.com/user-attachments/assets/ba86baab-38ec-45cb-9934-f04090a01f57

And this is what we see with reporter: [['default', { summary: false }]]

https://github.com/user-attachments/assets/ac0ebdf8-8513-4921-869d-ff19ca398a6c

In both cases we get a big blob of console output printed at the very end, because we pre-bundle our tests in a single file. That's the part we are trying to "hide".

On top of that, with summary: false there's also no sense of progress at all.

What we want is only the part that looks like this (is that the "summary"? I'm not sure):

 Test Files  1 passed (1)
      Tests  3971 passed (3971)
   Start at  15:29:30
   Duration  15.05s (transform 0ms, setup 15ms, collect 6.81s, tests 5.92s, environment 0ms, prepare 234ms)

But updated in real time, not just printed at the end.

[AriPerkkio]

Do you see expected output when you have 2 test files? For example by adding empty.test.ts with following:

import { test } from "vitest";

test("empty", () => {});

And when you have only 1 test file, do you see expected output with this kind of custom reporter:

import { DefaultReporter } from "vitest/reporters";

export default class CustomReporter extends DefaultReporter {
  onPathsCollected(paths) {
    super.onPathsCollected(paths);

    this.renderSucceed = false;
  }
}

I think this all might be related to weird logic of default reporter that I was afraid to remove. When there's only a single test file, the default reporter formats its output to match verbose reporter. We should probably remove this in next major release. 🤔

https://github.com/vitest-dev/vitest/blob/8b2b61f6fd9ab7eb41fa12df8c795d19cd65f175/packages/vitest/src/node/reporters/default.ts#L71-L81

https://github.com/vitest-dev/vitest/blob/8b2b61f6fd9ab7eb41fa12df8c795d19cd65f175/test/reporters/tests/default.test.ts#L59-L64

[acelaya]

Oh! So you mean there could be something here not working as expected? I opened this issue assuming everything was working as intended and therefore this would be a new feature request 😅

Let me try to add a dummy test file and see.

[acelaya]

Ok, with two test files the output of reporters: [['default', { summary: false }]] is equivalent to reporters: ['basic'], but it still does not provide any real-time feedback on individual tests, only on individual files.

[acelaya]

And when you have only 1 test file, do you see expected output with this kind of custom reporter:

import { DefaultReporter } from "vitest/reporters";

export default class CustomReporter extends DefaultReporter {
  onPathsCollected(paths) {
    super.onPathsCollected(paths);

    this.renderSucceed = false;
  }
}

Yep! This is the output we want 😄

https://github.com/user-attachments/assets/1c8f1398-5f71-47c4-bccd-57a70dfa9fd6

[AriPerkkio]

Ok, this makes sense now. This is not exactly a bug as it works exactly as intended, but I have no idea why it was built like this in the first place. I'll check this with rest of the team.

In my opinnion the default reporter should not act like verbose reporter, no matter how many test cases there are.

As work-around for now, you can either:

• Add second empty test file and use reporters: "default". Do not disable the summary
• Use the custom reporter that disables renderSucceed in onPathsCollected. Note that this approach uses Vitest's private APIs and can break in minor and patch releases.

[acelaya]

Thanks! I think for now we'll stick with our custom reporter which does not rely on any private API, only one public method.

[sheremet-va]

In my opinnion the default reporter should not act like verbose reporter, no matter how many test cases there are.

It does not act as a verbose reporter, it's a special behaviour of default reporter when you work with a single test in watch mode, for example, to see how tests are running. It exists for convenience (the example in this issue of having thousands of tests in a single file is an edge case that can be bypassed with a custom reporter). verbose reporter prints everything in every file, which is too much. See #3266

[sheremet-va]

As an additional note, at the time of implementing this feature, we didn't have a dynamic summary, so it might not be as relevant anymore, but jest does behave in the same way as Vitest does now.

[AriPerkkio]

it's a special behaviour of default reporter when you work with a single test in watch mode, for example, to see how tests are running. It exists for convenience (the example in this issue of having thousands of tests in a single file is an edge case that can be bypassed with a custom reporter).

Maybe this should only be enabled when filename is passed as filter and it matches only 1 file? That could even be just a patch fix instead of breaking change. 🤔

[sheremet-va]

We decided to avoid rendering succeeded tests that have more than the terminal rows of tests inside.