refactor: use jest test environment

[danielroe]

I'd be happy to refactor the way this module is invoked to use the Jest custom environment feature, which I think would be a more intuitive way of setting up tests.

For example, at the moment the setupTest approach breaks it.only for example, which might be non-intuitive to a person who doesn't understand what setupTest is doing under the hood. (Alternatively, the solution is being clearer in the documentation.)

However, because this would be a significant change, I wanted to get a :+1: from @pi0 and @ricardogobbosouza first. Thoughts?

[ricardogobbosouza]

Hi @danielroe Sorry, but i didn't understand what you intend to do 😅

[danielroe]

@ricardogobbosouza The normal jest environments are node and jsdom which provide appropriate globals for those environments.

I am proposing a nuxt environment (possibly more than one).

[ricardogobbosouza]

What we have at the moment is a preset, it would be incredible to have a environment for nuxt. My thoughts:

• we have to be careful that the environment is not too slow
• we also need an environment for the modules?

[vinayakkulkarni]

Any chance we could use ava instead of Facebook's Jest ?

[mrazauskas]

First of all, thank you for this library. After trying it out, I though it would work brilliantly as Jest environment. Happy to find this discussion.

Working Prototype

To try out quickly how @nuxt/test-utils environment could work I created this working prototype.

That is just a quickly written code. I was trying to make only minimal changes by wrapping Jest environment module around current API. Of course, this is clumsy, but both implementations work without conflicting.

Before diving into code or details I wanted to see the whole picture. What problems the code should solve? So I was concentrating on:

• usage examples
• documentation draft
• benchmark (see bellow)

These parts are the most important in the repo.

Proposal

The test environment could improve several things:

• simplify setup. Test environment declaration lets us know the exact structure of a project. nuxt-app points options.fixture to process.cwd(). nuxt-module looks for fixture(s) inside test/fixture or custom directories (current default);

• .only works as expected;

• slightly better performance (see Benchmark bellow).

Examples

Test environment can be declared by adding a @jest-environment docblock at the top of a test file or it can be defined through a jest.config.* file.

nuxt-app

/**
 * @jest-environment @nuxt/test-utils/nuxt-app
 */

describe('two features', () => {
  test('working feature', async () => {
    // ...
  })

  test.debug('failing feature', () => {
    // will open a browser for debugging
  })
})

// TODO `.debug` is not yet implemented

nuxt-module

// package.json

"test:e2e": "jest test/e2e -c jest.config.e2e.ts"

// jest.config.e2e.ts

const config: Config.InitialOptions = {
  // ...
  testEnvironment: '@nuxt/test-utils/nuxt-module'
}

// test/e2e/main.test.ts

setup({
  fixture: 'main-fixture',
  targets: ['server', 'static']
}).describe('with the main fixture', () => {
    test('first test', () => {
      // ...
    })

    setup({ configOverride: { ssr: false } })
      .test.only('second test', () => {
        // ...
      })
  })

// TODO `setup({}: NuxtConfig)` is not yet implemented

For more details see README inside the repo.

Benchmark

As I mentioned, the test environment is just an additional wrapper of current core implementation. Current @nuxt/test-utils code base was changed only minimally.

Each suite was ran ten times in similar conditions: one warm up run, headless browser, no linting. Here are the average run times:

• JavaScript example

  • Jest environment, 22.55s
  • setupTest, 25.77s

• TypeScript example

  • Jest environment, 33.15s
  • setupTest, 42.85s

• Robots Module

  • Jest environment, 58.95s
  • setupTest, 73.71s

Jest environment proofs to be ca. 10-20% faster. Both implementations are slow enough. That’s the nature of end-to-end tests.

Final Thoughts

I would be more than happy to work on this project. It was fun to research and to scratch the surface of Jest. Feels tempting to dive deeper.

Before moving on I would like to ask:

• does the proposed implementation look useful for you?
• do these ideas fit with your vision of test-utils?

Thanks for reading this longish text!

[ricardogobbosouza]

Amazing implementation @mrazauskas In my view, this super fits for this module 😄 I would like to know the opinion of @pi0 and @danielroe

[pi0]

Thanks for the detailed insights @mrazauskas. Of course, PR for supporting nuxt-app and nuxt-module environments is welcome. (only we need to implement them inside src as typescript. i can guide you then how to prepare packaging)

[mrazauskas]

Thanks for quick reply. I was not sure what should be the next step, but now everything got clear. I will work on the real implementation and will send a PR. Most probably that will not be very fast. I made a plan on how to implement everything. Let’s see.

(TypeScript and tests – these will be included for sure.)

[pi0]

thanks @mrazauskas. it would be nice if you can split your work as septate PRs so we can quickly iterate 😊

[mrazauskas]

Sure. That’s exactly what I was thinking. Also I see how few small optimisations could be made in the core. Will send these in while working on the environment.