es-toolkit icon indicating copy to clipboard operation
es-toolkit copied to clipboard

bench: change the benchmark tool from `vitest` to `tinybench`

Open dayongkr opened this issue 1 year ago • 2 comments

Description

Vitest introduces overhead from vite, which significantly impacts functions in utility tools with low computation, making it unsuitable for es-toolkit.

To address this, I implemented benchmarking using only tinybench. The process works as follows:

  1. When a user executes a command, the bench-runner.mjs script is triggered.
  2. This script compares the input file with files located in benchmarks/performance to determine which files to benchmark.
  3. The selected benchmark files are bundled with rollup and stored in the .bench-bundle folder.
  4. The bundled files are then executed using node <built-file>.
  5. This is repeated for all relevant files.

Additionally, to maintain the usability of vitest, I implemented the same describe and bench interface in the benchmark/bench.ts file.

The main differences are that, instead of importing fromvitest, the functions are now imported from ../bench, and the output is somewhat simpler. Other than that, commands are executed the same way, such as yarn bench chunk.

I’m sorry for opening such a large PR.🙏 If there are any questions or concerns, ask for a review at any time.

Screenshot 2024-10-14 at 4 06 23 PM Screenshot 2024-10-14 at 4 08 29 PM

I will gradually improve the output.

  • ✅ I ran through all the files.
  • ⚠️ But I didn’t compare them all against vitest again.

TODO

  • [X] Implement bundling scripts for es-toolkit and a benchmark file

Utilize the existing bundling setup to closely resemble the production environment.

  • [X] Abstract tinybench to have the same interface as vitest for convenient writing of benchmark code
  • [X] Implement the ability to run benchmarks via a command, specifying the file name
  • [X] Modify existing benchmark files:
    • [X] Remove reassigned parts
    • [X] Remove Vitest dependency

dayongkr avatar Oct 14 '24 04:10 dayongkr

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
es-toolkit ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 19, 2024 2:22pm

vercel[bot] avatar Oct 14 '24 04:10 vercel[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.03%. Comparing base (4b2de1f) to head (6c985f9).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #710   +/-   ##
=======================================
  Coverage   99.03%   99.03%           
=======================================
  Files         285      285           
  Lines        2483     2483           
  Branches      706      706           
=======================================
  Hits         2459     2459           
  Misses         23       23           
  Partials        1        1           

codecov-commenter avatar Oct 15 '24 04:10 codecov-commenter