fuels-ts icon indicating copy to clipboard operation
fuels-ts copied to clipboard

Move from Jest to a more suitable alternative

Open arboleya opened this issue 2 years ago • 10 comments

Although Jest does the job and has done it so far (although slower than I'd like) we should consider seeking a better alternative, possibly more modern and performant. Speed is paramount in tests, and Jest seems to fall short.

Vitest seems a strong candidate:

  • https://vitest.dev/guide/migration.html#migrating-from-jest

arboleya avatar Mar 05 '23 13:03 arboleya

@arboleya I agree. I will add something:

I feel like we should re-evaluate #728 since it has been blocked/stuck for a long time, and maybe just rename the issue to 'Switch to a new testing setup' and include switching to a new test runner in that issue. All of this feels like a part of one big problem that we are trying to solve.

We can first switch to a new test runner, and then look into how we can run those tests in a browser. Putting in more work into figuring out how to run the tests in a browser with Jest, only to switch to a different test runner soon doesn't sound ideal.

Dhaiwat10 avatar Mar 13 '23 15:03 Dhaiwat10

We need to first make sure vitest is compatible with Browser based testing and execution, IE Jasmine or Karma, or purely within the vistest ecosystem. see @vitest/browser (https://www.npmjs.com/package/@vitest/browser):

⚠️ This package is not yet ready and it's for preview only. While this package will be released along with other packages, it will not follow semver for breaking changes until we mark it as ready. Do not use it in production.

According to this GH issue it is NOT compatible, so that would be a hard blocker: https://github.com/vitest-dev/vitest/issues/586

But then later on I see this merge as it seems someone kept pushing, but not marked ready for production: https://github.com/vitest-dev/vitest/pull/1302

camsjams avatar Mar 13 '23 16:03 camsjams

Also I am not sure where we got the idea that vitest is faster, Jest is proven to be faster on multiple projects, so I'd like to see the benchmarks here.

See https://bradgarropy.com/blog/jest-over-vitest

I'd suggest we stay on Jest and just finish this PR https://github.com/FuelLabs/fuels-ts/pull/728 as is, and move on to the next thing

camsjams avatar Mar 13 '23 16:03 camsjams

@camsjams I went through that thread that you linked, someone suggested having a look at karma-vite

Dhaiwat10 avatar Mar 13 '23 16:03 Dhaiwat10

This comment also seems interesting, seems like this person is trying to achieve more or less what we want to: https://github.com/vitest-dev/vitest/issues/586#issuecomment-1396966245

Dhaiwat10 avatar Mar 13 '23 16:03 Dhaiwat10

@camsjams @Dhaiwat10 Yes, I agree that testing is delicate and that we should move on with #728 for now, which I'm working to unblock. That doesn't mean we can't talk about options here — btw, great set of links, guys. Thanks.

arboleya avatar Mar 13 '23 17:03 arboleya

Currently looking at implementing Vitest in #1310 and it is very quick compared to jest.

Duration 16.38s (transform 4.31s, setup 2.02s, collect 96.78s, tests 31.68s, environment 25ms, prepare 15.15s)

Still facing various issues with the browser implementation however that shouldn't stop us from using it for node environment tests if we wanted to.

danielbate avatar Oct 11 '23 13:10 danielbate

Given the performance updates and minimal interruption to our current workflows and tooling, we will implement Vitest completely for our node environment tests, and also add support for a browser test however this will not yet run in CI.

danielbate avatar Oct 18 '23 08:10 danielbate

@danielbate Am I tripping, or we're about to become 7.5x times faster (>10mins in CI) by moving to Vitest? ⚡🔥🤘

@camsjams I believe the above settles the matter. 😆

arboleya avatar Oct 20 '23 18:10 arboleya

@camsjams I believe the above settles the

@arboleya Benchmarks never lie *

But in this case, I agree. For the TS-SDK, the performance improvements are indeed valuable and clear gains 🎉

camsjams avatar Oct 20 '23 19:10 camsjams

Completed via https://github.com/FuelLabs/fuels-ts/pull/1310

arboleya avatar Jan 31 '24 20:01 arboleya