microsoft
Ensure that Comparers only return -1, 0, 1 so results can be equated
I noticed this while working on cleaning up our (inconsistent) use of the binary search and ordered insert APIs.
The binary search API requires that you have comparisons of exactly -1 | 0 | 1, however, we had some comparers which didn't do this thanks to a hole in our type system (Playground Link).
Make it so that we consistently return only one of the three valid values, so comparers can always be used in all APIs.
I also noticed (but did not change) the fact that we use binary searches for things like import insertion with the assumption that the input is always sorted, but, I don't think this actually turns out to be the case all the time anymore?
Heya @jakebailey, I've started to run the perf test suite on this PR at 2106d54771599640ddc50a4bdcdbabb8a35328bb. You can monitor the build here.
Update: The results are in!
Does this catch any actual bugs? Should we just change the comparer type to return number, and use Comparison as a set of easily accessible constants?
Does this catch any actual bugs? Should we just change the comparer type to return
number, and useComparisonas a set of easily accessible constants?
No, it doesn't, and that surprised me.
If we want to do that, the binary search API needs to change to not check equality on Comparison values. That could definitely be a good fix for this too.
Well, it did catch one bug; When I was working on #52891, I accidentally passed getTypeId as a Comparer, and had no idea what I had done wrong. I think that's the benefit here, but, I'm not strongly attached to it.
I think that if we had any existing bugs related to this, they'd already have been caught, because the result is pretty catastrophic.
@jakebailey The results of the perf run you requested are in!
Here they are:
Compiler
Comparison Report - main..52897| Metric | main | 52897 | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| Angular - node (v18.10.0, x64) | ||||||
| Memory used | 359,665k (± 0.01%) | 359,683k (± 0.02%) | ~ | 359,592k | 359,766k | p=0.689 n=6 |
| Parse Time | 3.49s (± 0.88%) | 3.48s (± 0.65%) | ~ | 3.45s | 3.51s | p=0.519 n=6 |
| Bind Time | 1.17s (± 0.64%) | 1.17s (± 1.03%) | ~ | 1.15s | 1.18s | p=0.485 n=6 |
| Check Time | 8.76s (± 0.26%) | 8.76s (± 0.34%) | ~ | 8.72s | 8.81s | p=0.732 n=6 |
| Emit Time | 7.61s (± 1.18%) | 7.61s (± 0.63%) | ~ | 7.54s | 7.68s | p=0.686 n=6 |
| Total Time | 21.02s (± 0.48%) | 21.02s (± 0.23%) | ~ | 20.96s | 21.10s | p=0.684 n=6 |
| Compiler-Unions - node (v18.10.0, x64) | ||||||
| Memory used | 193,630k (± 1.48%) | 191,839k (± 1.55%) | ~ | 189,848k | 195,726k | p=1.000 n=6 |
| Parse Time | 1.48s (± 0.66%) | 1.48s (± 0.28%) | ~ | 1.47s | 1.48s | p=0.930 n=6 |
| Bind Time | 0.80s (± 0.69%) | 0.80s (± 0.69%) | ~ | 0.79s | 0.80s | p=1.000 n=6 |
| Check Time | 9.45s (± 0.58%) | 9.46s (± 0.52%) | ~ | 9.41s | 9.53s | p=0.686 n=6 |
| Emit Time | 2.81s (± 1.13%) | 2.81s (± 0.66%) | ~ | 2.79s | 2.84s | p=0.570 n=6 |
| Total Time | 14.53s (± 0.48%) | 14.54s (± 0.31%) | ~ | 14.48s | 14.60s | p=0.936 n=6 |
| Monaco - node (v18.10.0, x64) | ||||||
| Memory used | 344,051k (± 0.02%) | 344,045k (± 0.02%) | ~ | 343,980k | 344,129k | p=0.810 n=6 |
| Parse Time | 2.62s (± 1.05%) | 2.63s (± 1.05%) | ~ | 2.59s | 2.66s | p=0.418 n=6 |
| Bind Time | 1.05s (± 0.80%) | 1.04s (± 0.95%) | ~ | 1.02s | 1.05s | p=0.340 n=6 |
| Check Time | 7.11s (± 0.25%) | 7.13s (± 0.46%) | ~ | 7.08s | 7.17s | p=0.195 n=6 |
| Emit Time | 4.34s (± 1.06%) | 4.33s (± 0.55%) | ~ | 4.29s | 4.36s | p=0.739 n=6 |
| Total Time | 15.12s (± 0.48%) | 15.13s (± 0.15%) | ~ | 15.11s | 15.17s | p=0.872 n=6 |
| TFS - node (v18.10.0, x64) | ||||||
| Memory used | 300,066k (± 0.00%) | 300,089k (± 0.01%) | ~ | 300,062k | 300,145k | p=0.054 n=6 |
| Parse Time | 2.08s (± 0.77%) | 2.04s (± 1.47%) | ~ | 2.01s | 2.08s | p=0.163 n=6 |
| Bind Time | 1.17s (± 0.76%) | 1.18s (± 1.03%) | ~ | 1.17s | 1.20s | p=0.437 n=6 |
| Check Time | 6.57s (± 0.22%) | 6.58s (± 0.28%) | ~ | 6.55s | 6.60s | p=0.808 n=6 |
| Emit Time | 3.92s (± 0.90%) | 3.90s (± 0.42%) | ~ | 3.88s | 3.92s | p=0.806 n=6 |
| Total Time | 13.74s (± 0.38%) | 13.70s (± 0.33%) | ~ | 13.62s | 13.75s | p=0.469 n=6 |
| material-ui - node (v18.10.0, x64) | ||||||
| Memory used | 476,437k (± 0.01%) | 476,400k (± 0.01%) | ~ | 476,373k | 476,453k | p=0.066 n=6 |
| Parse Time | 3.15s (± 0.52%) | 3.16s (± 0.24%) | ~ | 3.15s | 3.17s | p=0.216 n=6 |
| Bind Time | 0.97s (± 1.01%) | 0.97s (± 0.78%) | ~ | 0.96s | 0.98s | p=0.652 n=6 |
| Check Time | 17.30s (± 0.62%) | 17.23s (± 0.74%) | ~ | 17.04s | 17.40s | p=0.375 n=6 |
| Emit Time | 0.00s (± 0.00%) | 0.00s (± 0.00%) | ~ | 0.00s | 0.00s | p=1.000 n=6 |
| Total Time | 21.42s (± 0.57%) | 21.36s (± 0.61%) | ~ | 21.15s | 21.53s | p=0.575 n=6 |
| xstate - node (v18.10.0, x64) | ||||||
| Memory used | 548,584k (± 0.02%) | 548,509k (± 0.03%) | ~ | 548,328k | 548,809k | p=0.298 n=6 |
| Parse Time | 4.06s (± 0.55%) | 4.07s (± 0.40%) | ~ | 4.04s | 4.09s | p=0.332 n=6 |
| Bind Time | 1.71s (± 0.61%) | 1.72s (± 1.35%) | ~ | 1.69s | 1.75s | p=1.000 n=6 |
| Check Time | 2.83s (± 0.99%) | 2.82s (± 0.69%) | ~ | 2.79s | 2.84s | p=0.687 n=6 |
| Emit Time | 0.08s (± 0.00%) | 0.08s (± 6.19%) | ~ | 0.08s | 0.09s | p=0.176 n=6 |
| Total Time | 8.69s (± 0.40%) | 8.69s (± 0.21%) | ~ | 8.66s | 8.71s | p=0.935 n=6 |
| Angular - node (v16.17.1, x64) | ||||||
| Memory used | 359,020k (± 0.01%) | 359,064k (± 0.01%) | ~ | 359,010k | 359,124k | p=0.298 n=6 |
| Parse Time | 3.72s (± 0.43%) | 3.72s (± 0.49%) | ~ | 3.69s | 3.73s | p=0.870 n=6 |
| Bind Time | 1.23s (± 0.73%) | 1.23s (± 0.95%) | ~ | 1.22s | 1.25s | p=0.933 n=6 |
| Check Time | 9.55s (± 0.31%) | 9.52s (± 0.27%) | ~ | 9.48s | 9.55s | p=0.107 n=6 |
| Emit Time | 8.08s (± 0.44%) | 8.05s (± 0.53%) | ~ | 8.00s | 8.09s | p=0.373 n=6 |
| Total Time | 22.58s (± 0.27%) | 22.52s (± 0.18%) | ~ | 22.46s | 22.57s | p=0.127 n=6 |
| Compiler-Unions - node (v16.17.1, x64) | ||||||
| Memory used | 191,538k (± 0.03%) | 191,560k (± 0.04%) | ~ | 191,425k | 191,657k | p=0.471 n=6 |
| Parse Time | 1.54s (± 1.46%) | 1.57s (± 0.52%) | ~ | 1.55s | 1.57s | p=0.055 n=6 |
| Bind Time | 0.84s (± 0.61%) | 0.84s (± 0.00%) | ~ | 0.84s | 0.84s | p=0.174 n=6 |
| Check Time | 10.11s (± 0.47%) | 10.17s (± 0.27%) | ~ | 10.13s | 10.20s | p=0.064 n=6 |
| Emit Time | 3.05s (± 1.01%) | 3.06s (± 0.81%) | ~ | 3.04s | 3.11s | p=0.627 n=6 |
| Total Time | 15.54s (± 0.41%) | 15.63s (± 0.31%) | +0.09s (+ 0.59%) | 15.57s | 15.71s | p=0.030 n=6 |
| Monaco - node (v16.17.1, x64) | ||||||
| Memory used | 343,349k (± 0.00%) | 343,344k (± 0.00%) | ~ | 343,320k | 343,361k | p=0.375 n=6 |
| Parse Time | 2.80s (± 0.71%) | 2.80s (± 0.53%) | ~ | 2.78s | 2.82s | p=1.000 n=6 |
| Bind Time | 1.11s (± 0.37%) | 1.11s (± 0.37%) | ~ | 1.11s | 1.12s | p=1.000 n=6 |
| Check Time | 7.82s (± 0.36%) | 7.80s (± 0.20%) | ~ | 7.79s | 7.83s | p=0.145 n=6 |
| Emit Time | 4.52s (± 0.73%) | 4.52s (± 0.83%) | ~ | 4.48s | 4.57s | p=0.809 n=6 |
| Total Time | 16.25s (± 0.19%) | 16.23s (± 0.21%) | ~ | 16.19s | 16.28s | p=0.260 n=6 |
| TFS - node (v16.17.1, x64) | ||||||
| Memory used | 299,418k (± 0.00%) | 299,414k (± 0.01%) | ~ | 299,396k | 299,438k | p=0.689 n=6 |
| Parse Time | 2.20s (± 0.99%) | 2.19s (± 0.55%) | ~ | 2.18s | 2.21s | p=0.684 n=6 |
| Bind Time | 1.25s (± 0.67%) | 1.26s (± 0.41%) | ~ | 1.26s | 1.27s | p=0.073 n=6 |
| Check Time | 7.25s (± 0.33%) | 7.25s (± 0.32%) | ~ | 7.23s | 7.28s | p=0.871 n=6 |
| Emit Time | 4.24s (± 0.94%) | 4.23s (± 0.78%) | ~ | 4.20s | 4.29s | p=0.870 n=6 |
| Total Time | 14.94s (± 0.51%) | 14.94s (± 0.28%) | ~ | 14.89s | 14.99s | p=0.936 n=6 |
| material-ui - node (v16.17.1, x64) | ||||||
| Memory used | 475,699k (± 0.00%) | 475,693k (± 0.00%) | ~ | 475,652k | 475,713k | p=0.748 n=6 |
| Parse Time | 3.33s (± 0.29%) | 3.33s (± 0.25%) | ~ | 3.32s | 3.34s | p=0.195 n=6 |
| Bind Time | 1.02s (± 0.50%) | 1.02s (± 0.00%) | ~ | 1.02s | 1.02s | p=0.174 n=6 |
| Check Time | 18.28s (± 0.69%) | 18.20s (± 0.46%) | ~ | 18.12s | 18.35s | p=0.335 n=6 |
| Emit Time | 0.00s (± 0.00%) | 0.00s (± 0.00%) | ~ | 0.00s | 0.00s | p=1.000 n=6 |
| Total Time | 22.63s (± 0.57%) | 22.55s (± 0.38%) | ~ | 22.46s | 22.70s | p=0.261 n=6 |
| xstate - node (v16.17.1, x64) | ||||||
| Memory used | 546,132k (± 0.02%) | 546,130k (± 0.02%) | ~ | 546,018k | 546,277k | p=0.936 n=6 |
| Parse Time | 4.25s (± 0.27%) | 4.25s (± 0.39%) | ~ | 4.23s | 4.28s | p=0.681 n=6 |
| Bind Time | 1.81s (± 0.64%) | 1.81s (± 0.64%) | ~ | 1.80s | 1.83s | p=1.000 n=6 |
| Check Time | 3.06s (± 0.96%) | 3.07s (± 0.87%) | ~ | 3.02s | 3.09s | p=0.802 n=6 |
| Emit Time | 0.09s (± 0.00%) | 0.09s (± 0.00%) | ~ | 0.09s | 0.09s | p=1.000 n=6 |
| Total Time | 9.22s (± 0.42%) | 9.22s (± 0.37%) | ~ | 9.17s | 9.26s | p=1.000 n=6 |
| Angular - node (v14.15.1, x64) | ||||||
| Memory used | 352,589k (± 0.01%) | 352,586k (± 0.01%) | ~ | 352,562k | 352,637k | p=0.810 n=6 |
| Parse Time | 3.73s (± 0.26%) | 3.71s (± 0.33%) | -0.02s (- 0.45%) | 3.70s | 3.73s | p=0.027 n=6 |
| Bind Time | 1.28s (± 0.70%) | 1.28s (± 0.59%) | ~ | 1.27s | 1.29s | p=0.798 n=6 |
| Check Time | 9.84s (± 0.37%) | 9.84s (± 0.60%) | ~ | 9.78s | 9.95s | p=0.810 n=6 |
| Emit Time | 8.42s (± 0.77%) | 8.35s (± 0.39%) | ~ | 8.30s | 8.39s | p=0.252 n=6 |
| Total Time | 23.28s (± 0.17%) | 23.19s (± 0.35%) | ~ | 23.11s | 23.33s | p=0.109 n=6 |
| Compiler-Unions - node (v14.15.1, x64) | ||||||
| Memory used | 186,673k (± 0.01%) | 186,684k (± 0.02%) | ~ | 186,638k | 186,727k | p=0.521 n=6 |
| Parse Time | 1.58s (± 0.77%) | 1.58s (± 1.13%) | ~ | 1.56s | 1.61s | p=0.864 n=6 |
| Bind Time | 0.86s (± 0.88%) | 0.86s (± 0.95%) | ~ | 0.85s | 0.87s | p=0.729 n=6 |
| Check Time | 10.24s (± 0.76%) | 10.24s (± 0.74%) | ~ | 10.16s | 10.33s | p=0.809 n=6 |
| Emit Time | 3.18s (± 0.70%) | 3.17s (± 1.03%) | ~ | 3.13s | 3.22s | p=0.327 n=6 |
| Total Time | 15.85s (± 0.57%) | 15.84s (± 0.49%) | ~ | 15.76s | 15.96s | p=0.687 n=6 |
| Monaco - node (v14.15.1, x64) | ||||||
| Memory used | 338,190k (± 0.01%) | 338,189k (± 0.01%) | ~ | 338,162k | 338,212k | p=0.748 n=6 |
| Parse Time | 2.90s (± 0.31%) | 2.90s (± 0.48%) | ~ | 2.88s | 2.91s | p=0.562 n=6 |
| Bind Time | 1.12s (± 0.92%) | 1.12s (± 0.75%) | ~ | 1.10s | 1.12s | p=0.788 n=6 |
| Check Time | 8.15s (± 0.27%) | 8.13s (± 0.49%) | ~ | 8.08s | 8.18s | p=0.295 n=6 |
| Emit Time | 4.81s (± 0.46%) | 4.77s (± 0.45%) | -0.04s (- 0.83%) | 4.74s | 4.79s | p=0.016 n=6 |
| Total Time | 16.98s (± 0.20%) | 16.91s (± 0.32%) | -0.07s (- 0.42%) | 16.85s | 16.99s | p=0.037 n=6 |
| TFS - node (v14.15.1, x64) | ||||||
| Memory used | 294,346k (± 0.00%) | 294,351k (± 0.00%) | ~ | 294,341k | 294,375k | p=0.748 n=6 |
| Parse Time | 2.39s (± 0.62%) | 2.38s (± 0.82%) | ~ | 2.37s | 2.42s | p=0.459 n=6 |
| Bind Time | 1.10s (± 0.99%) | 1.10s (± 0.89%) | ~ | 1.09s | 1.11s | p=0.859 n=6 |
| Check Time | 7.53s (± 0.59%) | 7.54s (± 0.43%) | ~ | 7.49s | 7.59s | p=0.628 n=6 |
| Emit Time | 4.65s (± 0.77%) | 4.62s (± 1.17%) | ~ | 4.59s | 4.73s | p=0.145 n=6 |
| Total Time | 15.66s (± 0.31%) | 15.64s (± 0.20%) | ~ | 15.61s | 15.70s | p=0.469 n=6 |
| material-ui - node (v14.15.1, x64) | ||||||
| Memory used | 471,240k (± 0.00%) | 471,233k (± 0.01%) | ~ | 471,198k | 471,282k | p=0.377 n=6 |
| Parse Time | 3.49s (± 0.12%) | 3.49s (± 0.23%) | ~ | 3.48s | 3.50s | p=0.584 n=6 |
| Bind Time | 1.04s (± 0.61%) | 1.04s (± 1.05%) | ~ | 1.03s | 1.06s | p=0.787 n=6 |
| Check Time | 19.18s (± 0.57%) | 19.11s (± 0.43%) | ~ | 19.03s | 19.25s | p=0.261 n=6 |
| Emit Time | 0.00s (± 0.00%) | 0.00s (± 0.00%) | ~ | 0.00s | 0.00s | p=1.000 n=6 |
| Total Time | 23.72s (± 0.45%) | 23.65s (± 0.37%) | ~ | 23.55s | 23.79s | p=0.298 n=6 |
| xstate - node (v14.15.1, x64) | ||||||
| Memory used | 534,515k (± 0.01%) | 534,473k (± 0.02%) | ~ | 534,324k | 534,547k | p=0.689 n=6 |
| Parse Time | 4.60s (± 0.45%) | 4.59s (± 0.22%) | ~ | 4.58s | 4.61s | p=0.323 n=6 |
| Bind Time | 1.73s (± 0.48%) | 1.72s (± 0.30%) | ~ | 1.71s | 1.72s | p=0.073 n=6 |
| Check Time | 3.16s (± 0.73%) | 3.15s (± 0.68%) | ~ | 3.12s | 3.18s | p=0.520 n=6 |
| Emit Time | 0.10s (± 4.01%) | 0.10s (± 0.00%) | ~ | 0.10s | 0.10s | p=0.405 n=6 |
| Total Time | 9.60s (± 0.22%) | 9.57s (± 0.21%) | ~ | 9.55s | 9.60s | p=0.064 n=6 |
| Machine Name | ts-ci-ubuntu |
|---|---|
| Platform | linux 5.4.0-135-generic |
| Architecture | x64 |
| Available Memory | 16 GB |
| Available Memory | 15 GB |
| CPUs | 4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz |
- node (v18.10.0, x64)
- node (v16.17.1, x64)
- node (v14.15.1, x64)
- Angular - node (v18.10.0, x64)
- Angular - node (v16.17.1, x64)
- Angular - node (v14.15.1, x64)
- Compiler-Unions - node (v18.10.0, x64)
- Compiler-Unions - node (v16.17.1, x64)
- Compiler-Unions - node (v14.15.1, x64)
- Monaco - node (v18.10.0, x64)
- Monaco - node (v16.17.1, x64)
- Monaco - node (v14.15.1, x64)
- TFS - node (v18.10.0, x64)
- TFS - node (v16.17.1, x64)
- TFS - node (v14.15.1, x64)
- material-ui - node (v18.10.0, x64)
- material-ui - node (v16.17.1, x64)
- material-ui - node (v14.15.1, x64)
- xstate - node (v18.10.0, x64)
- xstate - node (v16.17.1, x64)
- xstate - node (v14.15.1, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | 52897 | 6 |
| Baseline | main | 6 |
TSServer
Comparison Report - main..52897| Metric | main | 52897 | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| Compiler-UnionsTSServer - node (v18.10.0, x64) | ||||||
| Req 1 - updateOpen | 2,380ms (± 0.38%) | 2,379ms (± 0.44%) | ~ | 2,367ms | 2,394ms | p=0.873 n=6 |
| Req 2 - geterr | 5,350ms (± 0.42%) | 5,383ms (± 0.31%) | +33ms (+ 0.62%) | 5,367ms | 5,406ms | p=0.020 n=6 |
| Req 3 - references | 342ms (± 1.42%) | 341ms (± 1.48%) | ~ | 335ms | 348ms | p=0.625 n=6 |
| Req 4 - navto | 282ms (± 0.62%) | 283ms (± 0.58%) | ~ | 280ms | 285ms | p=0.288 n=6 |
| Req 5 - completionInfo count | 1,356 (± 0.00%) | 1,356 (± 0.00%) | ~ | 1,356 | 1,356 | p=1.000 n=6 |
| Req 5 - completionInfo | 72ms (± 0.88%) | 72ms (± 0.57%) | ~ | 71ms | 72ms | p=0.673 n=6 |
| CompilerTSServer - node (v18.10.0, x64) | ||||||
| Req 1 - updateOpen | 2,504ms (± 0.74%) | 2,504ms (± 0.75%) | ~ | 2,475ms | 2,533ms | p=1.000 n=6 |
| Req 2 - geterr | 4,029ms (± 0.62%) | 4,039ms (± 0.82%) | ~ | 4,007ms | 4,073ms | p=0.630 n=6 |
| Req 3 - references | 350ms (± 0.97%) | 355ms (± 0.86%) | +5ms (+ 1.28%) | 353ms | 361ms | p=0.043 n=6 |
| Req 4 - navto | 290ms (± 0.42%) | 288ms (± 0.28%) | ~ | 287ms | 289ms | p=0.120 n=6 |
| Req 5 - completionInfo count | 1,518 (± 0.00%) | 1,518 (± 0.00%) | ~ | 1,518 | 1,518 | p=1.000 n=6 |
| Req 5 - completionInfo | 68ms (± 1.61%) | 67ms (± 0.77%) | ~ | 67ms | 68ms | p=0.247 n=6 |
| xstateTSServer - node (v18.10.0, x64) | ||||||
| Req 1 - updateOpen | 3,198ms (± 0.41%) | 3,196ms (± 0.34%) | ~ | 3,183ms | 3,212ms | p=0.810 n=6 |
| Req 2 - geterr | 1,638ms (± 1.17%) | 1,637ms (± 0.80%) | ~ | 1,621ms | 1,654ms | p=0.873 n=6 |
| Req 3 - references | 105ms (± 1.17%) | 105ms (± 1.17%) | ~ | 103ms | 106ms | p=1.000 n=6 |
| Req 4 - navto | 358ms (± 1.26%) | 357ms (± 0.57%) | ~ | 354ms | 360ms | p=1.000 n=6 |
| Req 5 - completionInfo count | 3,136 (± 0.00%) | 3,136 (± 0.00%) | ~ | 3,136 | 3,136 | p=1.000 n=6 |
| Req 5 - completionInfo | 426ms (± 1.53%) | 430ms (± 1.13%) | ~ | 424ms | 438ms | p=0.374 n=6 |
| Compiler-UnionsTSServer - node (v16.17.1, x64) | ||||||
| Req 1 - updateOpen | 2,518ms (± 1.00%) | 2,527ms (± 0.54%) | ~ | 2,515ms | 2,549ms | p=0.873 n=6 |
| Req 2 - geterr | 5,742ms (± 0.48%) | 5,765ms (± 0.26%) | ~ | 5,747ms | 5,784ms | p=0.173 n=6 |
| Req 3 - references | 354ms (± 1.22%) | 351ms (± 1.18%) | ~ | 343ms | 355ms | p=0.294 n=6 |
| Req 4 - navto | 280ms (± 0.89%) | 278ms (± 1.01%) | ~ | 273ms | 280ms | p=0.305 n=6 |
| Req 5 - completionInfo count | 1,356 (± 0.00%) | 1,356 (± 0.00%) | ~ | 1,356 | 1,356 | p=1.000 n=6 |
| Req 5 - completionInfo | 82ms (± 5.51%) | 82ms (± 4.25%) | ~ | 77ms | 86ms | p=0.806 n=6 |
| CompilerTSServer - node (v16.17.1, x64) | ||||||
| Req 1 - updateOpen | 2,688ms (± 0.61%) | 2,695ms (± 0.73%) | ~ | 2,664ms | 2,712ms | p=0.630 n=6 |
| Req 2 - geterr | 4,384ms (± 0.58%) | 4,375ms (± 0.38%) | ~ | 4,352ms | 4,401ms | p=0.748 n=6 |
| Req 3 - references | 363ms (± 0.86%) | 364ms (± 1.02%) | ~ | 357ms | 367ms | p=0.622 n=6 |
| Req 4 - navto | 287ms (± 1.14%) | 286ms (± 1.33%) | ~ | 280ms | 289ms | p=1.000 n=6 |
| Req 5 - completionInfo count | 1,518 (± 0.00%) | 1,518 (± 0.00%) | ~ | 1,518 | 1,518 | p=1.000 n=6 |
| Req 5 - completionInfo | 73ms (± 1.65%) | 73ms (± 0.87%) | ~ | 72ms | 74ms | p=0.673 n=6 |
| xstateTSServer - node (v16.17.1, x64) | ||||||
| Req 1 - updateOpen | 3,371ms (± 0.17%) | 3,370ms (± 0.60%) | ~ | 3,355ms | 3,410ms | p=0.229 n=6 |
| Req 2 - geterr | 1,864ms (± 5.71%) | 1,867ms (± 4.71%) | ~ | 1,803ms | 2,038ms | p=0.688 n=6 |
| Req 3 - references | 111ms (± 2.26%) | 113ms (± 2.65%) | ~ | 110ms | 118ms | p=0.191 n=6 |
| Req 4 - navto | 340ms (± 1.98%) | 345ms (± 2.16%) | ~ | 333ms | 353ms | p=0.295 n=6 |
| Req 5 - completionInfo count | 3,136 (± 0.00%) | 3,136 (± 0.00%) | ~ | 3,136 | 3,136 | p=1.000 n=6 |
| Req 5 - completionInfo | 434ms (± 0.94%) | 435ms (± 0.98%) | ~ | 430ms | 441ms | p=0.872 n=6 |
| Compiler-UnionsTSServer - node (v14.15.1, x64) | ||||||
| Req 1 - updateOpen | 2,617ms (± 0.65%) | 2,622ms (± 0.24%) | ~ | 2,610ms | 2,628ms | p=0.198 n=6 |
| Req 2 - geterr | 6,082ms (± 0.39%) | 6,083ms (± 0.42%) | ~ | 6,050ms | 6,119ms | p=1.000 n=6 |
| Req 3 - references | 365ms (± 0.35%) | 365ms (± 0.22%) | ~ | 364ms | 366ms | p=0.558 n=6 |
| Req 4 - navto | 279ms (± 1.01%) | 277ms (± 0.60%) | ~ | 275ms | 279ms | p=0.328 n=6 |
| Req 5 - completionInfo count | 1,356 (± 0.00%) | 1,356 (± 0.00%) | ~ | 1,356 | 1,356 | p=1.000 n=6 |
| Req 5 - completionInfo | 98ms (± 4.66%) | 101ms (± 0.54%) | ~ | 100ms | 101ms | p=0.201 n=6 |
| CompilerTSServer - node (v14.15.1, x64) | ||||||
| Req 1 - updateOpen | 2,843ms (± 0.85%) | 2,850ms (± 0.61%) | ~ | 2,823ms | 2,872ms | p=0.575 n=6 |
| Req 2 - geterr | 4,534ms (± 2.25%) | 4,621ms (± 2.50%) | ~ | 4,472ms | 4,714ms | p=0.128 n=6 |
| Req 3 - references | 378ms (± 0.94%) | 382ms (± 0.35%) | ~ | 380ms | 384ms | p=0.063 n=6 |
| Req 4 - navto | 296ms (± 1.20%) | 294ms (± 1.42%) | ~ | 290ms | 299ms | p=0.291 n=6 |
| Req 5 - completionInfo count | 1,518 (± 0.00%) | 1,518 (± 0.00%) | ~ | 1,518 | 1,518 | p=1.000 n=6 |
| Req 5 - completionInfo | 89ms (±10.97%) | 83ms (± 1.48%) | ~ | 81ms | 84ms | p=0.288 n=6 |
| xstateTSServer - node (v14.15.1, x64) | ||||||
| Req 1 - updateOpen | 3,698ms (± 0.76%) | 3,687ms (± 0.68%) | ~ | 3,646ms | 3,717ms | p=0.810 n=6 |
| Req 2 - geterr | 1,871ms (± 1.55%) | 1,851ms (± 2.23%) | ~ | 1,805ms | 1,908ms | p=0.423 n=6 |
| Req 3 - references | 128ms (± 3.39%) | 129ms (± 3.52%) | ~ | 124ms | 136ms | p=0.809 n=6 |
| Req 4 - navto | 376ms (± 0.85%) | 378ms (± 1.34%) | ~ | 372ms | 386ms | p=0.810 n=6 |
| Req 5 - completionInfo count | 3,136 (± 0.00%) | 3,136 (± 0.00%) | ~ | 3,136 | 3,136 | p=1.000 n=6 |
| Req 5 - completionInfo | 448ms (± 2.32%) | 447ms (± 1.59%) | ~ | 439ms | 459ms | p=0.630 n=6 |
| Machine Name | ts-ci-ubuntu |
|---|---|
| Platform | linux 5.4.0-135-generic |
| Architecture | x64 |
| Available Memory | 16 GB |
| Available Memory | 15 GB |
| CPUs | 4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz |
- node (v18.10.0, x64)
- node (v16.17.1, x64)
- node (v14.15.1, x64)
- Compiler-UnionsTSServer - node (v18.10.0, x64)
- Compiler-UnionsTSServer - node (v16.17.1, x64)
- Compiler-UnionsTSServer - node (v14.15.1, x64)
- CompilerTSServer - node (v18.10.0, x64)
- CompilerTSServer - node (v16.17.1, x64)
- CompilerTSServer - node (v14.15.1, x64)
- xstateTSServer - node (v18.10.0, x64)
- xstateTSServer - node (v16.17.1, x64)
- xstateTSServer - node (v14.15.1, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | 52897 | 6 |
| Baseline | main | 6 |
Startup
Comparison Report - main..52897| Metric | main | 52897 | Delta | Best | Worst | p-value |
|---|---|---|---|---|---|---|
| tsc-startup - node (v16.17.1, x64) | ||||||
| Execution time | 142.17ms (± 0.17%) | 143.27ms (± 0.23%) | +1.10ms (+ 0.78%) | 142.55ms | 149.55ms | p=0.000 n=600 |
| tsserver-startup - node (v16.17.1, x64) | ||||||
| Execution time | 226.17ms (± 0.15%) | 229.15ms (± 0.39%) | +2.98ms (+ 1.32%) | 226.88ms | 236.16ms | p=0.000 n=600 |
| tsserverlibrary-startup - node (v16.17.1, x64) | ||||||
| Execution time | 228.13ms (± 0.18%) | 230.23ms (± 0.31%) | +2.10ms (+ 0.92%) | 228.49ms | 236.22ms | p=0.000 n=600 |
| typescript-startup - node (v16.17.1, x64) | ||||||
| Execution time | 208.92ms (± 0.14%) | 210.72ms (± 0.23%) | +1.81ms (+ 0.87%) | 209.36ms | 215.10ms | p=0.000 n=600 |
| Machine Name | ts-ci-ubuntu |
|---|---|
| Platform | linux 5.4.0-135-generic |
| Architecture | x64 |
| Available Memory | 16 GB |
| Available Memory | 15 GB |
| CPUs | 4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz |
- node (v16.17.1, x64)
- tsc-startup - node (v16.17.1, x64)
- tsserver-startup - node (v16.17.1, x64)
- tsserverlibrary-startup - node (v16.17.1, x64)
- typescript-startup - node (v16.17.1, x64)
| Benchmark | Name | Iterations |
|---|---|---|
| Current | 52897 | 6 |
| Baseline | main | 6 |
Developer Information:
Interestingly, 5.0 nightly catches the case where the comparer returns a literal 2 but not the a - b case. Playground
Yeah, then then leftover one is sort of the "known hole" as it has uses. Unfortunate.
I've always wondered why we use Comparison rather than just number and do < 0/> 0/=== 0. Is there a performance benefit to using one or the other?
I also noticed (but did not change) the fact that we use binary searches for things like import insertion with the assumption that the input is always sorted, but, I don't think this actually turns out to be the case all the time anymore?
Are there places in the code where we're doing a binary search on imports that we haven't previously checked are sorted? IIRC, that's the purpose of detectSortCaseSensitivity.
We have public "branded" types like SortedReadonlyArray and SortedArray that are intended to be used to signal something is sorted, but I've never been a fan of them because they are only sorted in relation to a specific Comparer, which isn't captured in the type. While they may be "sorted" in one respect, you cannot always safely use them as an argument that accepts a "sorted" array without also providing the comparer that was used, which makes the branded type somewhat misleading.
I've always wondered why we use
Comparisonrather than justnumberand do< 0/> 0/=== 0. Is there a performance benefit to using one or the other?
I doubt it. But I think it's a little easier to understand.
While they may be "sorted" in one respect, you cannot always safely use them as an argument that accepts a "sorted" array without also providing the comparer that was used, which makes the branded type somewhat misleading.
Even so, I do like that even if you don't use the types always, you are forced to cast to a sorted array of you need to use them (depending), which does make one think twice about whether or not the array is actually sorted or not.
While they may be "sorted" in one respect, you cannot always safely use them as an argument that accepts a "sorted" array without also providing the comparer that was used, which makes the branded type somewhat misleading.
Even so, I do like that even if you don't use the types always, you are forced to cast to a sorted array of you need to use them (depending), which does make one think twice about whether or not the array is actually sorted or not.
I'd almost rather have a data structure that encapsulates both the sorted array and the comparer, though I'm not sure that's going to be as efficient.