TypeScript
TypeScript copied to clipboard
microsoft
perf: improve regexp performance by using non-capturing groups
This PR is a performance improvement to TS cli/compiler for RegExps of file path matching. Currently, it uses dynamically constructed regex with groups to match multiple path patterns. While all RegExp groups are capturing, but we only use regex.test(file) function, which doesn't consume those groups at all.
I am building a RegExp monitor regex-doctor and found those regexes are taking quite some time to do the matches, even in a very small project. I imagine the cost of running those regexes could scale up with the package size.
For example, in my testing repo, this regex cost a total 8.48ms to match against 17 paths (and crates a RegexMatchArray with 233 unconsumed groups in this case)
After this PR, the time cost to match with the exact same inputs becomes 3.36ms (60% faster on regex matching).
Can you help to initiate the bot to run a proper benchmark and see how it would help the overall performance? Thanks.
This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.
Funnily I was already about to send a PR similar to this one using the findings from https://www.npmjs.com/package/eslint-plugin-regexp.
@typescript-bot test it
Starting jobs; this comment will be updated as builds start and complete.
| Command | Status | Results |
|---|---|---|
test top400 |
✅ Started | ✅ Results |
user test this |
✅ Started | ✅ Results |
run dt |
✅ Started | ✅ Results |
perf test this faster |
✅ Started | 👀 Results |
Actually, this PR is perpendicular as it affects dynamically created regexes. Very nice!
@jakebailey Here are the results of running the user tests comparing main and refs/pull/58551/merge:
Everything looks good!
Hey @jakebailey, the results of running the DT tests are ready.
Everything looks the same!
@jakebailey The results of the perf run you requested are in!
Here they are:
tsc
Comparison Report - baseline..pr| Metric | baseline | pr | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| Compiler-Unions - node (v18.15.0, x64) | ||||||
| Errors | 30 | 30 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 62,154 | 62,154 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 50,248 | 50,248 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 193,485k (± 0.96%) | 193,449k (± 0.97%) | ~ | 192,199k | 195,880k | p=0.378 n=6 |
| Parse Time | 1.55s (± 1.62%) | 1.55s (± 1.98%) | ~ | 1.51s | 1.58s | p=0.745 n=6 |
| Bind Time | 0.86s (± 0.63%) | 0.87s (± 0.94%) | ~ | 0.86s | 0.88s | p=0.859 n=6 |
| Check Time | 11.31s (± 0.36%) | 11.34s (± 0.24%) | ~ | 11.32s | 11.39s | p=0.172 n=6 |
| Emit Time | 3.15s (± 0.43%) | 3.14s (± 0.79%) | ~ | 3.09s | 3.16s | p=0.934 n=6 |
| Total Time | 16.87s (± 0.24%) | 16.90s (± 0.17%) | ~ | 16.86s | 16.94s | p=0.198 n=6 |
| angular-1 - node (v18.15.0, x64) | ||||||
| Errors | 5 | 5 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 944,110 | 944,110 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 407,140 | 407,140 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 1,222,113k (± 0.00%) | 1,222,105k (± 0.00%) | ~ | 1,222,016k | 1,222,148k | p=0.936 n=6 |
| Parse Time | 6.77s (± 0.40%) | 6.79s (± 0.65%) | ~ | 6.73s | 6.84s | p=0.467 n=6 |
| Bind Time | 1.88s (± 0.58%) | 1.87s (± 0.52%) | ~ | 1.86s | 1.89s | p=0.143 n=6 |
| Check Time | 31.24s (± 0.52%) | 31.31s (± 0.28%) | ~ | 31.23s | 31.43s | p=0.423 n=6 |
| Emit Time | 14.71s (± 0.31%) | 14.71s (± 0.68%) | ~ | 14.58s | 14.84s | p=0.748 n=6 |
| Total Time | 54.59s (± 0.32%) | 54.69s (± 0.29%) | ~ | 54.43s | 54.92s | p=0.748 n=6 |
| mui-docs - node (v18.15.0, x64) | ||||||
| Errors | 5 | 5 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 1,964,178 | 1,964,178 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 819,287 | 819,287 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 1,849,617k (± 0.00%) | 1,849,640k (± 0.00%) | ~ | 1,849,605k | 1,849,691k | p=0.378 n=6 |
| Parse Time | 6.78s (± 0.49%) | 6.78s (± 0.28%) | ~ | 6.76s | 6.81s | p=0.808 n=6 |
| Bind Time | 2.29s (± 0.55%) | 2.30s (± 1.39%) | ~ | 2.27s | 2.36s | p=0.858 n=6 |
| Check Time | 58.56s (± 0.37%) | 58.92s (± 0.40%) | +0.36s (+ 0.62%) | 58.72s | 59.37s | p=0.031 n=6 |
| Emit Time | 0.14s | 0.14s (± 2.88%) | ~ | 0.14s | 0.15s | p=0.405 n=6 |
| Total Time | 67.77s (± 0.29%) | 68.13s (± 0.34%) | +0.37s (+ 0.54%) | 67.96s | 68.57s | p=0.031 n=6 |
| self-build-src - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 1,221,231 | 1,221,231 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 259,523 | 259,525 | +2 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
| Memory used | 2,337,530k (± 0.02%) | 2,362,787k (± 2.59%) | ~ | 2,336,844k | 2,487,867k | p=0.378 n=6 |
| Parse Time | 5.00s (± 1.14%) | 5.00s (± 0.90%) | ~ | 4.95s | 5.05s | p=0.810 n=6 |
| Bind Time | 1.88s (± 0.75%) | 1.88s (± 0.43%) | ~ | 1.87s | 1.89s | p=0.677 n=6 |
| Check Time | 33.78s (± 0.24%) | 33.80s (± 0.29%) | ~ | 33.66s | 33.91s | p=0.630 n=6 |
| Emit Time | 2.65s (± 1.60%) | 2.62s (± 2.99%) | ~ | 2.53s | 2.71s | p=0.748 n=6 |
| Total Time | 43.31s (± 0.34%) | 43.33s (± 0.37%) | ~ | 43.06s | 43.50s | p=0.748 n=6 |
| self-build-src-public-api - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 1,221,231 | 1,221,231 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 259,523 | 259,525 | +2 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
| Memory used | 2,413,565k (± 0.02%) | 2,413,646k (± 0.04%) | ~ | 2,412,411k | 2,414,535k | p=0.689 n=6 |
| Parse Time | 6.19s (± 0.79%) | 6.18s (± 1.14%) | ~ | 6.12s | 6.28s | p=0.575 n=6 |
| Bind Time | 2.00s (± 0.93%) | 2.03s (± 0.52%) | +0.02s (+ 1.00%) | 2.01s | 2.04s | p=0.040 n=6 |
| Check Time | 40.16s (± 0.25%) | 40.11s (± 0.34%) | ~ | 39.88s | 40.25s | p=0.521 n=6 |
| Emit Time | 3.11s (± 3.37%) | 3.12s (± 2.92%) | ~ | 3.00s | 3.26s | p=0.748 n=6 |
| Total Time | 51.48s (± 0.34%) | 51.46s (± 0.36%) | ~ | 51.22s | 51.73s | p=0.810 n=6 |
| self-compiler - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 256,768 | 256,768 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 104,587 | 104,589 | +2 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
| Memory used | 426,072k (± 0.01%) | 426,066k (± 0.01%) | ~ | 426,037k | 426,122k | p=0.810 n=6 |
| Parse Time | 2.79s (± 0.42%) | 2.79s (± 0.43%) | ~ | 2.78s | 2.81s | p=0.388 n=6 |
| Bind Time | 1.11s (± 0.74%) | 1.10s (± 0.74%) | ~ | 1.09s | 1.11s | p=0.389 n=6 |
| Check Time | 15.20s (± 0.49%) | 15.18s (± 0.44%) | ~ | 15.07s | 15.26s | p=0.747 n=6 |
| Emit Time | 1.15s (± 0.85%) | 1.15s (± 1.28%) | ~ | 1.13s | 1.17s | p=0.615 n=6 |
| Total Time | 20.25s (± 0.37%) | 20.22s (± 0.35%) | ~ | 20.11s | 20.30s | p=0.747 n=6 |
| ts-pre-modules - node (v18.15.0, x64) | ||||||
| Errors | 35 | 35 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 224,575 | 224,575 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 93,785 | 93,785 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 369,862k (± 0.03%) | 369,852k (± 0.03%) | ~ | 369,696k | 369,972k | p=0.810 n=6 |
| Parse Time | 3.53s (± 1.00%) | 3.51s (± 0.68%) | ~ | 3.48s | 3.55s | p=0.466 n=6 |
| Bind Time | 1.93s (± 0.71%) | 1.92s (± 1.52%) | ~ | 1.87s | 1.95s | p=0.560 n=6 |
| Check Time | 19.40s (± 0.26%) | 19.34s (± 0.19%) | ~ | 19.28s | 19.39s | p=0.053 n=6 |
| Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
| Total Time | 24.86s (± 0.28%) | 24.77s (± 0.26%) | -0.09s (- 0.38%) | 24.66s | 24.84s | p=0.045 n=6 |
| vscode - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 2,823,415 | 2,823,415 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 957,881 | 957,881 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 2,996,465k (± 0.00%) | 2,996,475k (± 0.00%) | ~ | 2,996,422k | 2,996,537k | p=0.936 n=6 |
| Parse Time | 17.05s (± 0.27%) | 17.03s (± 0.43%) | ~ | 16.94s | 17.16s | p=0.470 n=6 |
| Bind Time | 5.05s (± 0.35%) | 5.05s (± 0.41%) | ~ | 5.02s | 5.08s | p=0.681 n=6 |
| Check Time | 89.23s (± 0.19%) | 89.99s (± 2.25%) | ~ | 88.90s | 94.11s | p=1.000 n=6 |
| Emit Time | 28.86s (± 0.59%) | 28.03s (± 6.48%) | ~ | 24.32s | 28.88s | p=0.336 n=6 |
| Total Time | 140.19s (± 0.12%) | 140.09s (± 0.24%) | ~ | 139.65s | 140.50s | p=0.575 n=6 |
| webpack - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 265,866 | 265,866 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 108,401 | 108,401 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 410,578k (± 0.03%) | 410,488k (± 0.01%) | ~ | 410,426k | 410,528k | p=0.128 n=6 |
| Parse Time | 3.84s (± 1.07%) | 3.83s (± 0.96%) | ~ | 3.79s | 3.87s | p=1.000 n=6 |
| Bind Time | 1.67s (± 0.90%) | 1.66s (± 1.28%) | ~ | 1.63s | 1.69s | p=0.326 n=6 |
| Check Time | 17.00s (± 0.55%) | 16.91s (± 0.23%) | ~ | 16.86s | 16.97s | p=0.109 n=6 |
| Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
| Total Time | 22.51s (± 0.39%) | 22.41s (± 0.30%) | ~ | 22.34s | 22.49s | p=0.108 n=6 |
| xstate-main - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 524,639 | 524,639 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 178,906 | 178,906 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 462,703k (± 0.02%) | 462,732k (± 0.01%) | ~ | 462,667k | 462,837k | p=0.689 n=6 |
| Parse Time | 3.88s (± 0.55%) | 3.88s (± 0.47%) | ~ | 3.87s | 3.91s | p=0.676 n=6 |
| Bind Time | 1.44s (± 0.57%) | 1.45s (± 1.94%) | ~ | 1.41s | 1.47s | p=1.000 n=6 |
| Check Time | 22.44s (± 0.81%) | 22.47s (± 0.70%) | ~ | 22.26s | 22.73s | p=0.936 n=6 |
| Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
| Total Time | 27.76s (± 0.66%) | 27.80s (± 0.48%) | ~ | 27.60s | 28.02s | p=0.520 n=6 |
System info unknown
Hosts
- node (v18.15.0, x64)
Scenarios
- Compiler-Unions - node (v18.15.0, x64)
- angular-1 - node (v18.15.0, x64)
- mui-docs - node (v18.15.0, x64)
- self-build-src - node (v18.15.0, x64)
- self-build-src-public-api - node (v18.15.0, x64)
- self-compiler - node (v18.15.0, x64)
- ts-pre-modules - node (v18.15.0, x64)
- vscode - node (v18.15.0, x64)
- webpack - node (v18.15.0, x64)
- xstate-main - node (v18.15.0, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | pr | 6 |
| Baseline | baseline | 6 |
Developer Information:
@jakebailey Here are the results of running the top 400 repos comparing main and refs/pull/58551/merge:
Everything looks good!
Starting jobs; this comment will be updated as builds start and complete.
| Command | Status | Results |
|---|---|---|
perf test this |
✅ Started | 👀 Results |
@DanielRosenwasser The results of the perf run you requested are in!
Here they are:
tsc
Comparison Report - baseline..pr| Metric | baseline | pr | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| Compiler-Unions - node (v18.15.0, x64) | ||||||
| Errors | 30 | 30 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 62,153 | 62,153 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 50,242 | 50,242 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 192,910k (± 0.76%) | 194,723k (± 0.95%) | ~ | 192,309k | 196,020k | p=0.093 n=6 |
| Parse Time | 1.95s (± 1.71%) | 1.94s (± 1.00%) | ~ | 1.91s | 1.96s | p=0.466 n=6 |
| Bind Time | 1.07s (± 0.97%) | 1.06s (± 0.60%) | ~ | 1.05s | 1.07s | p=0.203 n=6 |
| Check Time | 14.03s (± 0.28%) | 14.00s (± 0.48%) | ~ | 13.90s | 14.06s | p=0.467 n=6 |
| Emit Time | 4.06s (± 0.69%) | 4.02s (± 0.85%) | -0.04s (- 1.03%) | 3.96s | 4.05s | p=0.030 n=6 |
| Total Time | 21.11s (± 0.34%) | 21.03s (± 0.25%) | ~ | 20.95s | 21.09s | p=0.078 n=6 |
| angular-1 - node (v18.15.0, x64) | ||||||
| Errors | 5 | 5 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 944,109 | 944,109 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 407,049 | 407,049 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 1,222,105k (± 0.00%) | 1,222,133k (± 0.00%) | ~ | 1,222,054k | 1,222,192k | p=0.378 n=6 |
| Parse Time | 8.11s (± 0.42%) | 8.12s (± 0.54%) | ~ | 8.06s | 8.17s | p=1.000 n=6 |
| Bind Time | 2.23s (± 0.62%) | 2.24s (± 0.73%) | ~ | 2.22s | 2.26s | p=1.000 n=6 |
| Check Time | 36.53s (± 0.69%) | 36.53s (± 0.29%) | ~ | 36.40s | 36.70s | p=0.689 n=6 |
| Emit Time | 18.02s (± 0.20%) | 18.08s (± 0.57%) | ~ | 17.96s | 18.26s | p=0.377 n=6 |
| Total Time | 64.91s (± 0.40%) | 64.96s (± 0.23%) | ~ | 64.73s | 65.12s | p=0.471 n=6 |
| mui-docs - node (v18.15.0, x64) | ||||||
| Errors | 5 | 5 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 2,054,061 | 2,054,061 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 899,820 | 899,820 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 2,067,163k (± 0.00%) | 2,067,127k (± 0.01%) | ~ | 2,066,913k | 2,067,273k | p=0.810 n=6 |
| Parse Time | 8.11s (± 0.42%) | 8.09s (± 0.28%) | ~ | 8.05s | 8.11s | p=0.227 n=6 |
| Bind Time | 2.74s (± 0.55%) | 2.74s (± 1.10%) | ~ | 2.71s | 2.79s | p=0.871 n=6 |
| Check Time | 81.80s (± 0.20%) | 81.47s (± 1.42%) | ~ | 79.13s | 82.23s | p=0.748 n=6 |
| Emit Time | 0.17s (± 3.32%) | 0.16s (± 3.16%) | ~ | 0.16s | 0.17s | p=0.640 n=6 |
| Total Time | 92.81s (± 0.19%) | 92.46s (± 1.23%) | ~ | 90.16s | 93.20s | p=0.936 n=6 |
| self-build-src - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 1,229,708 | 1,229,708 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 260,903 | 260,905 | +2 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
| Memory used | 2,376,233k (± 2.56%) | 2,377,416k (± 2.59%) | ~ | 2,351,728k | 2,503,442k | p=0.298 n=6 |
| Parse Time | 5.95s (± 1.21%) | 5.94s (± 1.47%) | ~ | 5.81s | 6.03s | p=0.936 n=6 |
| Bind Time | 2.27s (± 0.82%) | 2.26s (± 0.79%) | ~ | 2.23s | 2.28s | p=0.317 n=6 |
| Check Time | 40.15s (± 0.32%) | 40.03s (± 0.44%) | ~ | 39.76s | 40.28s | p=0.378 n=6 |
| Emit Time | 3.15s (± 5.55%) | 3.03s (± 3.07%) | ~ | 2.96s | 3.21s | p=0.093 n=6 |
| Total Time | 51.55s (± 0.61%) | 51.29s (± 0.42%) | ~ | 51.03s | 51.53s | p=0.173 n=6 |
| self-build-src-public-api - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 1,229,708 | 1,229,708 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 260,903 | 260,905 | +2 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
| Memory used | 2,477,724k (± 3.14%) | 2,527,916k (± 3.10%) | ~ | 2,425,984k | 2,579,187k | p=0.471 n=6 |
| Parse Time | 6.23s (± 0.76%) | 6.23s (± 0.85%) | ~ | 6.15s | 6.29s | p=1.000 n=6 |
| Bind Time | 2.02s (± 0.77%) | 2.03s (± 0.65%) | ~ | 2.01s | 2.04s | p=0.418 n=6 |
| Check Time | 40.50s (± 0.23%) | 40.59s (± 0.33%) | ~ | 40.46s | 40.83s | p=0.229 n=6 |
| Emit Time | 3.09s (± 2.95%) | 3.07s (± 1.70%) | ~ | 3.01s | 3.15s | p=1.000 n=6 |
| Total Time | 51.86s (± 0.23%) | 51.94s (± 0.33%) | ~ | 51.76s | 52.17s | p=0.575 n=6 |
| self-compiler - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 258,593 | 258,593 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 105,011 | 105,013 | +2 (+ 0.00%) | ~ | ~ | p=0.001 n=6 |
| Memory used | 428,267k (± 0.01%) | 428,266k (± 0.01%) | ~ | 428,230k | 428,312k | p=0.810 n=6 |
| Parse Time | 3.30s (± 0.83%) | 3.30s (± 0.80%) | ~ | 3.27s | 3.33s | p=1.000 n=6 |
| Bind Time | 1.33s (± 0.91%) | 1.31s (± 1.63%) | ~ | 1.27s | 1.33s | p=0.119 n=6 |
| Check Time | 18.07s (± 0.43%) | 18.09s (± 0.35%) | ~ | 18.02s | 18.17s | p=0.687 n=6 |
| Emit Time | 1.36s (± 1.58%) | 1.37s (± 1.30%) | ~ | 1.34s | 1.39s | p=0.517 n=6 |
| Total Time | 24.06s (± 0.36%) | 24.07s (± 0.24%) | ~ | 24.00s | 24.17s | p=0.687 n=6 |
| ts-pre-modules - node (v18.15.0, x64) | ||||||
| Errors | 35 | 35 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 224,565 | 224,565 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 93,734 | 93,734 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 369,926k (± 0.04%) | 369,888k (± 0.04%) | ~ | 369,716k | 370,152k | p=0.471 n=6 |
| Parse Time | 2.85s (± 0.87%) | 2.84s (± 0.62%) | ~ | 2.81s | 2.86s | p=0.226 n=6 |
| Bind Time | 1.58s (± 0.86%) | 1.59s (± 1.05%) | ~ | 1.57s | 1.61s | p=0.560 n=6 |
| Check Time | 15.76s (± 0.25%) | 15.66s (± 0.24%) | -0.09s (- 0.59%) | 15.61s | 15.70s | p=0.005 n=6 |
| Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
| Total Time | 20.19s (± 0.19%) | 20.09s (± 0.15%) | -0.10s (- 0.52%) | 20.06s | 20.12s | p=0.005 n=6 |
| vscode - node (v18.15.0, x64) | ||||||
| Errors | 10 | 10 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 2,843,982 | 2,843,982 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 963,683 | 963,683 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 3,016,424k (± 0.00%) | 3,016,483k (± 0.00%) | ~ | 3,016,443k | 3,016,566k | p=0.066 n=6 |
| Parse Time | 13.90s (± 0.26%) | 13.86s (± 0.17%) | -0.04s (- 0.28%) | 13.83s | 13.89s | p=0.044 n=6 |
| Bind Time | 4.16s (± 0.35%) | 4.18s (± 0.61%) | ~ | 4.16s | 4.23s | p=0.304 n=6 |
| Check Time | 73.98s (± 2.17%) | 73.92s (± 2.14%) | ~ | 73.17s | 77.15s | p=0.228 n=6 |
| Emit Time | 23.12s (± 6.75%) | 23.09s (± 6.92%) | ~ | 19.83s | 23.91s | p=0.575 n=6 |
| Total Time | 115.16s (± 0.23%) | 115.05s (± 0.14%) | ~ | 114.78s | 115.24s | p=0.148 n=6 |
| webpack - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 266,361 | 266,361 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 108,554 | 108,554 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 411,296k (± 0.01%) | 411,360k (± 0.03%) | ~ | 411,249k | 411,618k | p=0.471 n=6 |
| Parse Time | 3.23s (± 0.58%) | 3.22s (± 0.25%) | ~ | 3.21s | 3.23s | p=0.617 n=6 |
| Bind Time | 1.41s (± 1.00%) | 1.40s (± 0.58%) | ~ | 1.39s | 1.41s | p=0.098 n=6 |
| Check Time | 14.47s (± 0.33%) | 14.52s (± 0.34%) | ~ | 14.45s | 14.58s | p=0.147 n=6 |
| Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
| Total Time | 19.10s (± 0.37%) | 19.14s (± 0.31%) | ~ | 19.05s | 19.20s | p=0.378 n=6 |
| xstate-main - node (v18.15.0, x64) | ||||||
| Errors | 0 | 0 | ~ | ~ | ~ | p=1.000 n=6 |
| Symbols | 526,436 | 526,436 | ~ | ~ | ~ | p=1.000 n=6 |
| Types | 179,425 | 179,425 | ~ | ~ | ~ | p=1.000 n=6 |
| Memory used | 464,047k (± 0.10%) | 464,163k (± 0.07%) | ~ | 463,495k | 464,355k | p=0.810 n=6 |
| Parse Time | 2.63s (± 0.21%) | 2.64s (± 0.39%) | ~ | 2.62s | 2.65s | p=0.663 n=6 |
| Bind Time | 1.00s (± 0.41%) | 1.00s (± 0.51%) | ~ | 1.00s | 1.01s | p=0.114 n=6 |
| Check Time | 15.30s (± 0.30%) | 15.38s (± 0.92%) | ~ | 15.25s | 15.64s | p=0.378 n=6 |
| Emit Time | 0.00s | 0.00s | ~ | ~ | ~ | p=1.000 n=6 |
| Total Time | 18.94s (± 0.25%) | 19.02s (± 0.74%) | ~ | 18.87s | 19.27s | p=0.297 n=6 |
System info unknown
Hosts
- node (v18.15.0, x64)
Scenarios
- Compiler-Unions - node (v18.15.0, x64)
- angular-1 - node (v18.15.0, x64)
- mui-docs - node (v18.15.0, x64)
- self-build-src - node (v18.15.0, x64)
- self-build-src-public-api - node (v18.15.0, x64)
- self-compiler - node (v18.15.0, x64)
- ts-pre-modules - node (v18.15.0, x64)
- vscode - node (v18.15.0, x64)
- webpack - node (v18.15.0, x64)
- xstate-main - node (v18.15.0, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | pr | 6 |
| Baseline | baseline | 6 |
tsserver
Comparison Report - baseline..pr| Metric | baseline | pr | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| Compiler-UnionsTSServer - node (v18.15.0, x64) | ||||||
| Req 1 - updateOpen | 3,409ms (± 0.33%) | 3,418ms (± 0.38%) | ~ | 3,403ms | 3,434ms | p=0.423 n=6 |
| Req 2 - geterr | 7,589ms (± 0.65%) | 7,567ms (± 0.49%) | ~ | 7,500ms | 7,601ms | p=0.575 n=6 |
| Req 3 - references | 434ms (± 1.21%) | 434ms (± 1.80%) | ~ | 427ms | 446ms | p=1.000 n=6 |
| Req 4 - navto | 337ms (± 0.46%) | 336ms (± 0.76%) | ~ | 332ms | 338ms | p=0.796 n=6 |
| Req 5 - completionInfo count | 1,357 | 1,357 | ~ | ~ | ~ | p=1.000 n=6 |
| Req 5 - completionInfo | 117ms (± 5.91%) | 120ms (± 6.47%) | ~ | 111ms | 128ms | p=0.411 n=6 |
| CompilerTSServer - node (v18.15.0, x64) | ||||||
| Req 1 - updateOpen | 2,463ms (± 0.62%) | 2,459ms (± 0.65%) | ~ | 2,443ms | 2,480ms | p=0.630 n=6 |
| Req 2 - geterr | 3,903ms (± 0.33%) | 3,903ms (± 0.25%) | ~ | 3,888ms | 3,914ms | p=0.810 n=6 |
| Req 3 - references | 298ms (± 0.39%) | 297ms (± 0.62%) | ~ | 294ms | 299ms | p=0.357 n=6 |
| Req 4 - navto | 229ms (± 0.24%) | 231ms (± 2.55%) | ~ | 228ms | 243ms | p=0.476 n=6 |
| Req 5 - completionInfo count | 1,519 | 1,519 | ~ | ~ | ~ | p=1.000 n=6 |
| Req 5 - completionInfo | 78ms (± 7.85%) | 76ms (± 9.10%) | ~ | 67ms | 84ms | p=0.805 n=6 |
| xstate-main-1-tsserver - node (v18.15.0, x64) | ||||||
| Req 1 - updateOpen | 5,135ms (± 0.55%) | 5,113ms (± 0.39%) | ~ | 5,080ms | 5,134ms | p=0.173 n=6 |
| Req 2 - geterr | 1,136ms (± 1.45%) | 1,123ms (± 1.40%) | ~ | 1,093ms | 1,137ms | p=0.230 n=6 |
| Req 3 - references | 95ms (± 4.99%) | 98ms (± 2.17%) | ~ | 95ms | 100ms | p=0.167 n=6 |
| Req 4 - navto | 453ms (± 0.59%) | 455ms (± 0.73%) | ~ | 450ms | 460ms | p=0.418 n=6 |
| Req 5 - completionInfo count | 3,413 | 3,413 | ~ | ~ | ~ | p=1.000 n=6 |
| Req 5 - completionInfo | 846ms (± 2.51%) | 841ms (± 0.52%) | ~ | 835ms | 846ms | p=1.000 n=6 |
System info unknown
Hosts
- node (v18.15.0, x64)
Scenarios
- CompilerTSServer - node (v18.15.0, x64)
- Compiler-UnionsTSServer - node (v18.15.0, x64)
- xstate-main-1-tsserver - node (v18.15.0, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | pr | 6 |
| Baseline | baseline | 6 |
startup
Comparison Report - baseline..pr| Metric | baseline | pr | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| tsc-startup - node (v18.15.0, x64) | ||||||
| Execution time | 159.27ms (± 0.16%) | 159.18ms (± 0.14%) | -0.09ms (- 0.06%) | 158.11ms | 160.52ms | p=0.003 n=600 |
| tsserver-startup - node (v18.15.0, x64) | ||||||
| Execution time | 239.82ms (± 0.12%) | 239.87ms (± 0.15%) | ~ | 238.39ms | 248.20ms | p=0.284 n=600 |
| tsserverlibrary-startup - node (v18.15.0, x64) | ||||||
| Execution time | 240.38ms (± 0.14%) | 240.35ms (± 0.14%) | ~ | 238.98ms | 243.64ms | p=0.481 n=600 |
| typescript-startup - node (v18.15.0, x64) | ||||||
| Execution time | 237.41ms (± 0.17%) | 237.44ms (± 0.15%) | ~ | 235.82ms | 241.99ms | p=0.291 n=600 |
System info unknown
Hosts
- node (v18.15.0, x64)
Scenarios
- tsc-startup - node (v18.15.0, x64)
- tsserver-startup - node (v18.15.0, x64)
- tsserverlibrary-startup - node (v18.15.0, x64)
- typescript-startup - node (v18.15.0, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | pr | 6 |
| Baseline | baseline | 6 |
Developer Information:
While I do not fully understand how to read the report, I see it mostly reduces time by a tiny fraction ~0.5%, which I think is still valuable considering the change is straightforward. Meanwhile, if you decide that this is not significant to have, also feel free to close the PR, no worries at all
In #59099 I ended up doing some of the same work (for some reason I thought I was in a different section of the codebase), and also largely didn't get a perf boost.
I don't know if we think it makes sense to bring this in, but if it does, there are a few extra places that I managed to catch in #59099 if you want to bring those into this PR too.
I see it mostly reduces time by a tiny fraction ~0.5%, which I think is still valuable considering the change is straightforward
That's for one benchmark out of many; we have so many benchmarks that we are guaranteed to have at least one or two spuriously show as statistically significant (we're p hacking ourselves).
That's not to say this PR doesn't have value, just that the difference may not actually be measurable for most people.