TypeScript Allow some per file compiler options

In this PR, we allow certain compiler options to be configured via comments within a file for the scope of that file using a comment of the form // @ts-option [value]. This means you can write, for example: compiler-option-pragmas

Currently this supports everything in the "strict type checks" and "additional checks" segments of options, which includes:

strict
noImplicitAny
strictNullChecks
strictFunctionTypes
strictBindCallApply
strictPropertyInitialization
noImplicitThis
useUnknownInCatchVariables
alwaysStrict
noUnusedLocals
noUnusedParameters
exactOptionalPropertyTypes
noImplicitReturns
noFallthroughCasesInSwitch
noUncheckedIndexedAccess
noImplicitOverride
noPropertyAccessFromIndexSignature

Adding more options to this list is just a matter of adding the allowedAsPragma flag to the definition of the option in commandLineParser.ts and using getFileLocalCompilerOption to look up the option at relevant locations, so we can easily add support for more local options as needed (by far the hardest was strictNullChecks).

Fixes #31035

I've wanted to look into this since I originally added the generic pragma-parsing infrastructure for @ts-check and related comments.

To start off the discussion, in the linked issue, @RyanCavanaugh asked:

It's not really even clear what this means - if you have a type declared in file A, another type in file B, and you try to relate them in file C, whose value of strictFunctionTypes is in play?

Currently in this PR, whichever is stricter. If either file the types originated from has it set, it's related strictly. Could go the other way and relate loosely if either is explicitly non-strict if looser checking is more preferable. Honestly, most of these "which context controls the flag" already have fairly canonical answers, either because we already look up a context to check JS-y-ness (for JS-file-specific-behaviors or error ignoring), or to issue an error in (in which case the error node location is a pretty logical context).

Anyways, I'm hoping this implementation can jumpstart discussion on the value of in-file pragmas for compiler options, specifically with the goal of bikeshedding the syntax and what options we should ultimately expose on a per-file basis. (Technically, I've tried to make the internal APIs for adding new per-file pragmas very type-safe and very easy to use.)

I strongly recommend reviewing this by-commit if possible. Specifically, the changes to strictNullChecks to support this are extensive and are contained in https://github.com/microsoft/TypeScript/pull/49886/commits/f5134a545c5ab657087828e92c3999dc02af037d.

Jul 12 '22 23:07 weswigham

@typescript-bot pack this

Jul 12 '22 23:07 weswigham

Heya @weswigham, I've started to run the tarball bundle task on this PR at 459196b1bd08f1e762f1c7deb855813222cdd476. You can monitor the build here.

Jul 12 '22 23:07 typescript-bot

@typescript-bot pack this now that baselines should be green again :)

Jul 13 '22 00:07 weswigham

Heya @weswigham, I've started to run the tarball bundle task on this PR at 97bba40778b7b4b92f40c0a22171c5b92dab346b. You can monitor the build here.

Jul 13 '22 00:07 typescript-bot

Is this opt-in only, or will it be possible to opt-out as well?

Jul 13 '22 06:07 MartinJohns

You should be able to write things like // @ts-strict false and the like. Parameters get parsed the same way they do on the command line.

Jul 13 '22 07:07 weswigham

@typescript-bot pack this

Jul 13 '22 19:07 weswigham

Heya @weswigham, I've started to run the tarball bundle task on this PR at ef3813c3f531b8bc2fd27f2305feddcdc2bc96a2. You can monitor the build here.

Jul 13 '22 19:07 typescript-bot

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/129711/artifacts?artifactName=tgz&fileId=07F2813C225AA6CC371299076DC3BF08FFE79A5C45BC00165AB9D270F70FC3FB02&fileName=/typescript-4.8.0-insiders.20220713.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

Jul 13 '22 20:07 typescript-bot

This should be cleaned up and ready to go, so let's get the bot to do some tests on this iteration (not the much has changed implementation-wise since I ran them on the strict null checks-dedicated draft pr).

@typescript-bot pack this @typescript-bot user test this inline @typescript-bot test this @typescript-bot run dt @typescript-bot perf test this

Aug 31 '22 18:08 weswigham

Heya @weswigham, I've started to run the extended test suite on this PR at 4bde2e71da10b1abf04f99de9bd935448696fd6c. You can monitor the build here.

Aug 31 '22 18:08 typescript-bot

Heya @weswigham, I've started to run the perf test suite on this PR at 4bde2e71da10b1abf04f99de9bd935448696fd6c. You can monitor the build here.

Update: The results are in!

Aug 31 '22 18:08 typescript-bot

Heya @weswigham, I've started to run the tarball bundle task on this PR at 4bde2e71da10b1abf04f99de9bd935448696fd6c. You can monitor the build here.

Aug 31 '22 18:08 typescript-bot

Heya @weswigham, I've started to run the diff-based user code test suite on this PR at 4bde2e71da10b1abf04f99de9bd935448696fd6c. You can monitor the build here.

Update: The results are in!

Aug 31 '22 18:08 typescript-bot

Heya @weswigham, I've started to run the parallelized Definitely Typed test suite on this PR at 4bde2e71da10b1abf04f99de9bd935448696fd6c. You can monitor the build here.

Aug 31 '22 18:08 typescript-bot

@weswigham Here are the results of running the user test suite comparing main and refs/pull/49886/merge:

Something interesting changed - please have a look.

Details

webpack

tsconfig.json

[NEW] error TS2322: Type '{ timestamp: boolean; hash: boolean; } | { timestamp: boolean; hash?: undefined; }' is not assignable to type '{ hash?: boolean; timestamp?: boolean; }'.
- lib/config/defaults.js#L444
- lib/config/defaults.js#L447
[NEW] error TS2769: No overload matches this call.
- lib/dependencies/HarmonyExportInitFragment.js#L71
- lib/dependencies/HarmonyExportInitFragment.js#L85
[NEW] error TS2322: Type 'undefined' is not assignable to type '(value: any) => void'.
- lib/serialization/FileMiddleware.js#L587
[NEW] error TS2322: Type 'undefined' is not assignable to type '(reason?: any) => void'.
- lib/serialization/FileMiddleware.js#L588

tsconfig.types.json

[NEW] error TS2322: Type 'undefined' is not assignable to type 'Pick<ArgumentConfig, "values" | "type">'.
- lib/cli.js#L167
[NEW] error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Record<"filter" | "sort" | "result" | "extract" | "filterSorted" | "groupResults" | "sortResults" | "filterResults" | "merge" | "getItemName" | "getItemFactory", Map<string, SyncBailHook<[...], any, UnsetAdditionalOptions>[]>>'.
- lib/stats/StatsFactory.js#L77
- lib/stats/StatsFactory.js#L144
[MISSING] error TS2322: Type 'undefined' is not assignable to type 'Pick<ArgumentConfig, "type" | "values">'.
- lib/cli.js#L167
[MISSING] error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Record<"sort" | "filter" | "result" | "extract" | "filterSorted" | "groupResults" | "sortResults" | "filterResults" | "merge" | "getItemName" | "getItemFactory", Map<string, SyncBailHook<[...], any, UnsetAdditionalOptions>[]>>'.
- lib/stats/StatsFactory.js#L77
- lib/stats/StatsFactory.js#L144

Aug 31 '22 18:08 typescript-bot

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/133185/artifacts?artifactName=tgz&fileId=99738104B9F8C6122A08A47F6E4906E162B58E2CD37725A530DD9E94E0D93EF802&fileName=/typescript-4.9.0-insiders.20220831.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

Aug 31 '22 18:08 typescript-bot

Heya @weswigham, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

Aug 31 '22 18:08 typescript-bot

@weswigham The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..49886

_Metric	_main	₄₉₈₈₆	_Delta	_Best	_Worst
_{Angular - node (v10.16.3, x64)}
_{Memory used}	_{363,148k (± 0.02%)}	_{367,453k (± 0.01%)}	_{+4,305k (+ 1.19%)}	_367,353k	_367,584k
_{Parse Time}	_{2.10s (± 0.64%)}	_{2.09s (± 0.58%)}	_{-0.01s (- 0.48%)}	_2.07s	_2.13s
_{Bind Time}	_{0.79s (± 0.84%)}	_{0.79s (± 0.71%)}	_{-0.00s (- 0.25%)}	_0.78s	_0.80s
_{Check Time}	_{6.18s (± 0.69%)}	_{6.67s (± 0.64%)}	_{+0.50s (+ 8.06%)}	_6.59s	_6.77s
_{Emit Time}	_{6.07s (± 0.87%)}	_{6.03s (± 0.72%)}	_{-0.04s (- 0.58%)}	_5.95s	_6.17s
_{Total Time}	_{15.13s (± 0.53%)}	_{15.58s (± 0.46%)}	_{+0.45s (+ 3.00%)}	_15.46s	_15.78s
_{Compiler-Unions - node (v10.16.3, x64)}
_{Memory used}	_{206,058k (± 0.04%)}	_{208,961k (± 0.36%)}	_{+2,903k (+ 1.41%)}	_208,395k	_212,005k
_{Parse Time}	_{0.84s (± 1.01%)}	_{0.84s (± 0.53%)}	_{+0.00s (+ 0.36%)}	_0.83s	_0.85s
_{Bind Time}	_{0.47s (± 1.04%)}	_{0.47s (± 1.11%)}	_{-0.00s (- 1.05%)}	_0.46s	_0.48s
_{Check Time}	_{8.47s (± 0.21%)}	_{9.01s (± 0.49%)}	_{+0.54s (+ 6.40%)}	_8.87s	_9.08s
_{Emit Time}	_{2.50s (± 1.53%)}	_{2.49s (± 1.05%)}	_{-0.01s (- 0.40%)}	_2.43s	_2.54s
_{Total Time}	_{12.29s (± 0.29%)}	_{12.82s (± 0.46%)}	_{+0.53s (+ 4.30%)}	_12.62s	_12.91s
_{Monaco - node (v10.16.3, x64)}
_{Memory used}	_{344,739k (± 0.03%)}	_{351,990k (± 0.02%)}	_{+7,251k (+ 2.10%)}	_351,840k	_352,086k
_{Parse Time}	_{1.61s (± 0.74%)}	_{1.60s (± 0.42%)}	_{-0.01s (- 0.50%)}	_1.58s	_1.61s
_{Bind Time}	_{0.70s (± 0.63%)}	_{0.70s (± 0.49%)}	_{-0.01s (- 0.71%)}	_0.69s	_0.70s
_{Check Time}	_{6.26s (± 0.39%)}	_{6.83s (± 0.50%)}	_{+0.57s (+ 9.07%)}	_6.76s	_6.92s
_{Emit Time}	_{3.23s (± 0.99%)}	_{3.21s (± 0.84%)}	_{-0.02s (- 0.74%)}	_3.16s	_3.28s
_{Total Time}	_{11.81s (± 0.43%)}	_{12.34s (± 0.46%)}	_{+0.53s (+ 4.51%)}	_12.23s	_12.49s
_{TFS - node (v10.16.3, x64)}
_{Memory used}	_{305,921k (± 0.01%)}	_{311,656k (± 0.03%)}	_{+5,735k (+ 1.87%)}	_311,471k	_311,847k
_{Parse Time}	_{1.29s (± 0.70%)}	_{1.29s (± 0.38%)}	_{-0.00s (- 0.23%)}	_1.28s	_1.30s
_{Bind Time}	_{0.68s (± 0.77%)}	_{0.67s (± 0.92%)}	_{-0.01s (- 1.18%)}	_0.66s	_0.68s
_{Check Time}	_{5.51s (± 0.68%)}	_{6.02s (± 0.44%)}	_{+0.51s (+ 9.22%)}	_5.96s	_6.08s
_{Emit Time}	_{3.42s (± 2.11%)}	_{3.36s (± 0.79%)}	_{-0.06s (- 1.61%)}	_3.31s	_3.42s
_{Total Time}	_{10.90s (± 0.83%)}	_{11.33s (± 0.34%)}	_{+0.44s (+ 4.01%)}	_11.22s	_11.42s
_{material-ui - node (v10.16.3, x64)}
_{Memory used}	_{460,722k (± 0.01%)}	_{463,132k (± 0.02%)}	_{+2,411k (+ 0.52%)}	_462,972k	_463,355k
_{Parse Time}	_{1.83s (± 0.82%)}	_{1.82s (± 0.58%)}	_{-0.01s (- 0.66%)}	_1.80s	_1.85s
_{Bind Time}	_{0.58s (± 1.79%)}	_{0.58s (± 1.67%)}	_{-0.00s (- 0.52%)}	_0.56s	_0.59s
_{Check Time}	_{15.81s (± 0.76%)}	_{17.10s (± 0.79%)}	_{+1.29s (+ 8.14%)}	_16.85s	_17.37s
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_{0.00s ( NaN%)}	_0.00s	_0.00s
_{Total Time}	_{18.23s (± 0.73%)}	_{19.50s (± 0.68%)}	_{+1.27s (+ 6.96%)}	_19.24s	_19.78s
_{xstate - node (v10.16.3, x64)}
_{Memory used}	_{587,723k (± 1.25%)}	_{588,988k (± 1.26%)}	_{+1,264k (+ 0.22%)}	_585,545k	_618,876k
_{Parse Time}	_{2.62s (± 0.47%)}	_{2.60s (± 0.49%)}	_{-0.02s (- 0.92%)}	_2.57s	_2.63s
_{Bind Time}	_{0.92s (± 0.67%)}	_{0.91s (± 0.52%)}	_{-0.01s (- 0.87%)}	_0.90s	_0.92s
_{Check Time}	_{1.63s (± 0.47%)}	_{1.71s (± 0.56%)}	_{+0.08s (+ 5.10%)}	_1.69s	_1.72s
_{Emit Time}	_{0.07s (± 4.13%)}	_{0.07s (± 4.92%)}	_{+0.00s (+ 2.78%)}	_0.07s	_0.08s
_{Total Time}	_{5.24s (± 0.27%)}	_{5.28s (± 0.35%)}	_{+0.04s (+ 0.78%)}	_5.25s	_5.33s
_{Angular - node (v12.1.0, x64)}
_{Memory used}	_{340,592k (± 0.09%)}	_{344,878k (± 0.08%)}	_{+4,286k (+ 1.26%)}	_343,788k	_345,195k
_{Parse Time}	_{2.08s (± 0.57%)}	_{2.09s (± 0.99%)}	_{+0.01s (+ 0.43%)}	_2.05s	_2.14s
_{Bind Time}	_{0.77s (± 0.58%)}	_{0.77s (± 0.68%)}	_{+0.00s (+ 0.13%)}	_0.76s	_0.78s
_{Check Time}	_{5.83s (± 0.64%)}	_{6.31s (± 0.65%)}	_{+0.48s (+ 8.21%)}	_6.24s	_6.43s
_{Emit Time}	_{6.25s (± 0.80%)}	_{6.23s (± 0.80%)}	_{-0.02s (- 0.34%)}	_6.13s	_6.35s
_{Total Time}	_{14.93s (± 0.47%)}	_{15.39s (± 0.59%)}	_{+0.47s (+ 3.14%)}	_15.23s	_15.65s
_{Compiler-Unions - node (v12.1.0, x64)}
_{Memory used}	_{193,897k (± 0.12%)}	_{196,376k (± 0.12%)}	_{+2,479k (+ 1.28%)}	_195,712k	_196,679k
_{Parse Time}	_{0.83s (± 0.99%)}	_{0.83s (± 0.63%)}	_{-0.01s (- 0.72%)}	_0.82s	_0.84s
_{Bind Time}	_{0.47s (± 1.25%)}	_{0.47s (± 0.71%)}	_{-0.00s (- 0.84%)}	_0.46s	_0.48s
_{Check Time}	_{6.68s (± 0.40%)}	_{7.18s (± 0.65%)}	_{+0.50s (+ 7.42%)}	_7.09s	_7.31s
_{Emit Time}	_{2.47s (± 1.87%)}	_{2.46s (± 0.88%)}	_{-0.01s (- 0.36%)}	_2.42s	_2.52s
_{Total Time}	_{10.46s (± 0.56%)}	_{10.94s (± 0.53%)}	_{+0.48s (+ 4.56%)}	_10.83s	_11.09s
_{Monaco - node (v12.1.0, x64)}
_{Memory used}	_{327,692k (± 0.01%)}	_{334,957k (± 0.02%)}	_{+7,264k (+ 2.22%)}	_334,757k	_335,054k
_{Parse Time}	_{1.58s (± 0.97%)}	_{1.57s (± 0.73%)}	_{-0.01s (- 0.82%)}	_1.54s	_1.60s
_{Bind Time}	_{0.70s (± 0.93%)}	_{0.69s (± 0.71%)}	_{-0.00s (- 0.57%)}	_0.69s	_0.71s
_{Check Time}	_{5.85s (± 0.47%)}	_{6.38s (± 0.33%)}	_{+0.53s (+ 9.08%)}	_6.33s	_6.43s
_{Emit Time}	_{3.30s (± 0.74%)}	_{3.28s (± 0.56%)}	_{-0.02s (- 0.52%)}	_3.23s	_3.32s
_{Total Time}	_{11.43s (± 0.45%)}	_{11.92s (± 0.29%)}	_{+0.50s (+ 4.34%)}	_11.85s	_11.98s
_{TFS - node (v12.1.0, x64)}
_{Memory used}	_{290,427k (± 0.08%)}	_{296,224k (± 0.01%)}	_{+5,797k (+ 2.00%)}	_296,174k	_296,309k
_{Parse Time}	_{1.30s (± 1.03%)}	_{1.29s (± 1.03%)}	_{-0.00s (- 0.31%)}	_1.27s	_1.33s
_{Bind Time}	_{0.68s (± 1.18%)}	_{0.67s (± 0.99%)}	_{-0.00s (- 0.74%)}	_0.66s	_0.69s
_{Check Time}	_{5.40s (± 0.57%)}	_{5.86s (± 0.48%)}	_{+0.46s (+ 8.56%)}	_5.81s	_5.92s
_{Emit Time}	_{3.51s (± 0.87%)}	_{3.47s (± 0.80%)}	_{-0.04s (- 1.08%)}	_3.42s	_3.54s
_{Total Time}	_{10.88s (± 0.53%)}	_{11.30s (± 0.46%)}	_{+0.42s (+ 3.84%)}	_11.19s	_11.45s
_{material-ui - node (v12.1.0, x64)}
_{Memory used}	_{439,760k (± 0.09%)}	_{442,037k (± 0.09%)}	_{+2,277k (+ 0.52%)}	_440,987k	_442,547k
_{Parse Time}	_{1.82s (± 0.92%)}	_{1.81s (± 0.49%)}	_{-0.01s (- 0.60%)}	_1.79s	_1.83s
_{Bind Time}	_{0.56s (± 1.07%)}	_{0.55s (± 0.73%)}	_{-0.01s (- 1.26%)}	_0.54s	_0.56s
_{Check Time}	_{12.90s (± 1.45%)}	_{14.20s (± 0.93%)}	_{+1.31s (+10.12%)}	_13.99s	_14.54s
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_{0.00s ( NaN%)}	_0.00s	_0.00s
_{Total Time}	_{15.27s (± 1.32%)}	_{16.57s (± 0.80%)}	_{+1.29s (+ 8.47%)}	_16.36s	_16.89s
_{xstate - node (v12.1.0, x64)}
_{Memory used}	_{552,857k (± 1.28%)}	_{550,986k (± 0.01%)}	_{-1,871k (- 0.34%)}	_550,823k	_551,185k
_{Parse Time}	_{2.55s (± 0.65%)}	_{2.54s (± 0.54%)}	_{-0.01s (- 0.39%)}	_2.52s	_2.57s
_{Bind Time}	_{0.89s (± 0.65%)}	_{0.89s (± 1.15%)}	_{-0.00s (- 0.11%)}	_0.87s	_0.92s
_{Check Time}	_{1.49s (± 0.57%)}	_{1.56s (± 0.33%)}	_{+0.07s (+ 4.56%)}	_1.55s	_1.57s
_{Emit Time}	_{0.07s (± 0.00%)}	_{0.07s (± 0.00%)}	_{0.00s ( 0.00%)}	_0.07s	_0.07s
_{Total Time}	_{5.00s (± 0.48%)}	_{5.05s (± 0.41%)}	_{+0.04s (+ 0.90%)}	_5.02s	_5.10s
_{Angular - node (v14.15.1, x64)}
_{Memory used}	_{338,732k (± 0.00%)}	_{342,827k (± 0.01%)}	_{+4,095k (+ 1.21%)}	_342,787k	_342,860k
_{Parse Time}	_{2.08s (± 1.37%)}	_{2.06s (± 0.84%)}	_{-0.02s (- 0.91%)}	_2.04s	_2.11s
_{Bind Time}	_{0.79s (± 0.38%)}	_{0.80s (± 0.47%)}	_{+0.00s (+ 0.38%)}	_0.79s	_0.80s
_{Check Time}	_{5.82s (± 0.45%)}	_{6.34s (± 0.52%)}	_{+0.52s (+ 8.93%)}	_6.26s	_6.42s
_{Emit Time}	_{6.14s (± 0.63%)}	_{6.20s (± 0.55%)}	_{+0.05s (+ 0.90%)}	_6.12s	_6.28s
_{Total Time}	_{14.84s (± 0.46%)}	_{15.40s (± 0.31%)}	_{+0.56s (+ 3.79%)}	_15.30s	_15.52s
_{Compiler-Unions - node (v14.15.1, x64)}
_{Memory used}	_{192,635k (± 0.01%)}	_{195,495k (± 0.38%)}	_{+2,860k (+ 1.48%)}	_195,118k	_198,478k
_{Parse Time}	_{0.86s (± 0.78%)}	_{0.85s (± 0.47%)}	_{-0.01s (- 0.93%)}	_0.84s	_0.86s
_{Bind Time}	_{0.49s (± 0.91%)}	_{0.49s (± 0.77%)}	_{-0.00s (- 0.61%)}	_0.48s	_0.49s
_{Check Time}	_{6.69s (± 0.41%)}	_{7.25s (± 0.30%)}	_{+0.56s (+ 8.32%)}	_7.20s	_7.31s
_{Emit Time}	_{2.39s (± 0.46%)}	_{2.42s (± 0.48%)}	_{+0.02s (+ 0.96%)}	_2.40s	_2.45s
_{Total Time}	_{10.43s (± 0.28%)}	_{11.00s (± 0.24%)}	_{+0.57s (+ 5.47%)}	_10.95s	_11.06s
_{Monaco - node (v14.15.1, x64)}
_{Memory used}	_{326,520k (± 0.01%)}	_{333,749k (± 0.01%)}	_{+7,229k (+ 2.21%)}	_333,695k	_333,787k
_{Parse Time}	_{1.58s (± 0.64%)}	_{1.57s (± 0.62%)}	_{-0.01s (- 0.38%)}	_1.55s	_1.59s
_{Bind Time}	_{0.72s (± 0.77%)}	_{0.72s (± 0.97%)}	_{-0.00s (- 0.42%)}	_0.71s	_0.74s
_{Check Time}	_{5.72s (± 0.28%)}	_{6.35s (± 0.62%)}	_{+0.63s (+11.03%)}	_6.25s	_6.42s
_{Emit Time}	_{3.32s (± 0.72%)}	_{3.33s (± 0.71%)}	_{+0.02s (+ 0.57%)}	_3.28s	_3.37s
_{Total Time}	_{11.33s (± 0.25%)}	_{11.97s (± 0.47%)}	_{+0.64s (+ 5.66%)}	_11.82s	_12.07s
_{TFS - node (v14.15.1, x64)}
_{Memory used}	_{289,622k (± 0.01%)}	_{295,300k (± 0.01%)}	_{+5,678k (+ 1.96%)}	_295,263k	_295,356k
_{Parse Time}	_{1.30s (± 0.73%)}	_{1.31s (± 0.89%)}	_{+0.00s (+ 0.08%)}	_1.29s	_1.33s
_{Bind Time}	_{0.79s (± 0.78%)}	_{0.79s (± 0.70%)}	_{+0.00s (+ 0.13%)}	_0.78s	_0.81s
_{Check Time}	_{5.34s (± 0.39%)}	_{5.88s (± 0.36%)}	_{+0.54s (+10.07%)}	_5.85s	_5.95s
_{Emit Time}	_{3.60s (± 0.64%)}	_{3.56s (± 0.48%)}	_{-0.04s (- 1.06%)}	_3.53s	_3.59s
_{Total Time}	_{11.03s (± 0.37%)}	_{11.53s (± 0.33%)}	_{+0.50s (+ 4.53%)}	_11.47s	_11.65s
_{material-ui - node (v14.15.1, x64)}
_{Memory used}	_{438,198k (± 0.00%)}	_{440,591k (± 0.01%)}	_{+2,393k (+ 0.55%)}	_440,529k	_440,666k
_{Parse Time}	_{1.86s (± 0.40%)}	_{1.85s (± 0.72%)}	_{-0.00s (- 0.22%)}	_1.83s	_1.88s
_{Bind Time}	_{0.58s (± 0.82%)}	_{0.59s (± 1.14%)}	_{+0.00s (+ 0.69%)}	_0.58s	_0.61s
_{Check Time}	_{12.90s (± 0.52%)}	_{14.26s (± 0.52%)}	_{+1.36s (+10.53%)}	_14.12s	_14.49s
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_{0.00s ( NaN%)}	_0.00s	_0.00s
_{Total Time}	_{15.34s (± 0.43%)}	_{16.70s (± 0.45%)}	_{+1.36s (+ 8.85%)}	_16.58s	_16.94s
_{xstate - node (v14.15.1, x64)}
_{Memory used}	_{547,392k (± 0.00%)}	_{548,711k (± 0.00%)}	_{+1,319k (+ 0.24%)}	_548,670k	_548,751k
_{Parse Time}	_{2.60s (± 0.32%)}	_{2.60s (± 0.46%)}	_{-0.00s (- 0.08%)}	_2.57s	_2.63s
_{Bind Time}	_{0.97s (± 0.77%)}	_{0.97s (± 1.12%)}	_{-0.00s (- 0.10%)}	_0.96s	_1.01s
_{Check Time}	_{1.54s (± 0.54%)}	_{1.62s (± 0.51%)}	_{+0.09s (+ 5.73%)}	_1.61s	_1.64s
_{Emit Time}	_{0.07s (± 3.14%)}	_{0.07s (± 3.14%)}	_{0.00s ( 0.00%)}	_0.07s	_0.08s
_{Total Time}	_{5.19s (± 0.30%)}	_{5.26s (± 0.26%)}	_{+0.08s (+ 1.50%)}	_5.23s	_5.30s

System

_{Machine Name}	_ts-ci-ubuntu
_Platform	_{linux 4.4.0-210-generic}
_Architecture	_x64
_{Available Memory}	_{16 GB}
_{Available Memory}	_{15 GB}
_CPUs	_{4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz}

Hosts

node (v10.16.3, x64)
node (v12.1.0, x64)
node (v14.15.1, x64)

Scenarios

Angular - node (v10.16.3, x64)
Angular - node (v12.1.0, x64)
Angular - node (v14.15.1, x64)
Compiler-Unions - node (v10.16.3, x64)
Compiler-Unions - node (v12.1.0, x64)
Compiler-Unions - node (v14.15.1, x64)
Monaco - node (v10.16.3, x64)
Monaco - node (v12.1.0, x64)
Monaco - node (v14.15.1, x64)
TFS - node (v10.16.3, x64)
TFS - node (v12.1.0, x64)
TFS - node (v14.15.1, x64)
material-ui - node (v10.16.3, x64)
material-ui - node (v12.1.0, x64)
material-ui - node (v14.15.1, x64)
xstate - node (v10.16.3, x64)
xstate - node (v12.1.0, x64)
xstate - node (v14.15.1, x64)

_Benchmark	_Name	_Iterations
_Current	₄₉₈₈₆	₁₀
_Baseline	_main	₁₀

TSServer

Comparison Report - main..49886

_Metric	_main	₄₉₈₈₆	_Delta	_Best	_Worst
_{Compiler-UnionsTSServer - node (v10.16.3, x64)}
_{Req 1 - updateOpen}	_{1,408ms (± 0.38%)}	_{1,394ms (± 0.44%)}	_{-14ms (- 1.02%)}	_1,382ms	_1,408ms
_{Req 2 - geterr}	_{4,116ms (± 0.43%)}	_{4,377ms (± 0.64%)}	_{+262ms (+ 6.35%)}	_4,308ms	_4,427ms
_{Req 3 - references}	_{231ms (± 0.77%)}	_{232ms (± 0.66%)}	_{+1ms (+ 0.35%)}	_228ms	_236ms
_{Req 4 - navto}	_{172ms (± 1.12%)}	_{170ms (± 0.49%)}	_{-3ms (- 1.57%)}	_167ms	_171ms
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_{0 ( 0.00%)}	_1,356	_1,356
_{Req 5 - completionInfo}	_{92ms (± 6.61%)}	_{87ms (± 7.01%)}	_{🟩-6ms (- 6.07%)}	_67ms	_97ms
_{CompilerTSServer - node (v10.16.3, x64)}
_{Req 1 - updateOpen}	_{1,502ms (± 0.49%)}	_{1,492ms (± 0.38%)}	_{-9ms (- 0.61%)}	_1,472ms	_1,500ms
_{Req 2 - geterr}	_{2,255ms (± 0.49%)}	_{2,420ms (± 0.38%)}	_{+165ms (+ 7.29%)}	_2,398ms	_2,436ms
_{Req 3 - references}	_{242ms (± 0.61%)}	_{244ms (± 0.72%)}	_{+2ms (+ 0.66%)}	_238ms	_246ms
_{Req 4 - navto}	_{184ms (± 1.12%)}	_{183ms (± 0.69%)}	_{-2ms (- 0.92%)}	_180ms	_186ms
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_{0 ( 0.00%)}	_1,518	_1,518
_{Req 5 - completionInfo}	_{69ms (± 0.84%)}	_{68ms (± 2.82%)}	_{-1ms (- 1.46%)}	_63ms	_70ms
_{xstateTSServer - node (v10.16.3, x64)}
_{Req 1 - updateOpen}	_{2,095ms (± 0.40%)}	_{2,076ms (± 0.38%)}	_{-20ms (- 0.94%)}	_2,056ms	_2,090ms
_{Req 2 - geterr}	_{804ms (± 0.67%)}	_{842ms (± 0.61%)}	_{+38ms (+ 4.69%)}	_832ms	_851ms
_{Req 3 - references}	_{100ms (± 1.96%)}	_{100ms (± 2.14%)}	_{-1ms (- 0.80%)}	_95ms	_103ms
_{Req 4 - navto}	_{232ms (± 0.74%)}	_{234ms (± 0.98%)}	_{+1ms (+ 0.60%)}	_229ms	_238ms
_{Req 5 - completionInfo count}	_{3,301 (± 0.00%)}	_{3,301 (± 0.00%)}	_{0 ( 0.00%)}	_3,301	_3,301
_{Req 5 - completionInfo}	_{283ms (± 0.65%)}	_{280ms (± 1.19%)}	_{-3ms (- 1.06%)}	_273ms	_287ms
_{Compiler-UnionsTSServer - node (v12.1.0, x64)}
_{Req 1 - updateOpen}	_{1,404ms (± 0.69%)}	_{1,400ms (± 0.58%)}	_{-5ms (- 0.33%)}	_1,378ms	_1,416ms
_{Req 2 - geterr}	_{3,372ms (± 0.38%)}	_{3,605ms (± 0.35%)}	_{+233ms (+ 6.90%)}	_3,585ms	_3,633ms
_{Req 3 - references}	_{219ms (± 0.91%)}	_{220ms (± 1.18%)}	_{+1ms (+ 0.23%)}	_216ms	_226ms
_{Req 4 - navto}	_{161ms (± 1.14%)}	_{161ms (± 0.94%)}	_{-0ms (- 0.19%)}	_157ms	_163ms
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_{0 ( 0.00%)}	_1,356	_1,356
_{Req 5 - completionInfo}	_{60ms (± 5.54%)}	_{61ms (± 9.68%)}	_{+1ms (+ 1.68%)}	_55ms	_84ms
_{CompilerTSServer - node (v12.1.0, x64)}
_{Req 1 - updateOpen}	_{1,493ms (± 0.67%)}	_{1,495ms (± 0.88%)}	_{+2ms (+ 0.10%)}	_1,472ms	_1,524ms
_{Req 2 - geterr}	_{2,164ms (± 0.50%)}	_{2,324ms (± 0.43%)}	_{+160ms (+ 7.37%)}	_2,300ms	_2,340ms
_{Req 3 - references}	_{232ms (± 0.80%)}	_{234ms (± 0.74%)}	_{+2ms (+ 0.95%)}	_231ms	_238ms
_{Req 4 - navto}	_{172ms (± 1.30%)}	_{172ms (± 1.17%)}	_{+1ms (+ 0.35%)}	_167ms	_177ms
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_{0 ( 0.00%)}	_1,518	_1,518
_{Req 5 - completionInfo}	_{63ms (± 3.71%)}	_{60ms (± 2.87%)}	_{🟩-3ms (- 4.58%)}	_58ms	_66ms
_{xstateTSServer - node (v12.1.0, x64)}
_{Req 1 - updateOpen}	_{2,038ms (± 0.66%)}	_{2,027ms (± 0.49%)}	_{-11ms (- 0.53%)}	_2,003ms	_2,053ms
_{Req 2 - geterr}	_{751ms (± 0.44%)}	_{788ms (± 0.56%)}	_{+37ms (+ 4.97%)}	_780ms	_801ms
_{Req 3 - references}	_{67ms (± 0.92%)}	_{67ms (± 1.40%)}	_{+0ms (+ 0.60%)}	_65ms	_69ms
_{Req 4 - navto}	_{224ms (± 1.83%)}	_{224ms (± 1.42%)}	_{+1ms (+ 0.27%)}	_214ms	_230ms
_{Req 5 - completionInfo count}	_{3,301 (± 0.00%)}	_{3,301 (± 0.00%)}	_{0 ( 0.00%)}	_3,301	_3,301
_{Req 5 - completionInfo}	_{273ms (± 1.21%)}	_{271ms (± 0.81%)}	_{-2ms (- 0.66%)}	_267ms	_278ms
_{Compiler-UnionsTSServer - node (v14.15.1, x64)}
_{Req 1 - updateOpen}	_{1,441ms (± 0.43%)}	_{1,444ms (± 0.77%)}	_{+3ms (+ 0.24%)}	_1,429ms	_1,482ms
_{Req 2 - geterr}	_{3,489ms (± 0.41%)}	_{3,762ms (± 0.35%)}	_{+273ms (+ 7.82%)}	_3,730ms	_3,778ms
_{Req 3 - references}	_{229ms (± 0.66%)}	_{229ms (± 0.63%)}	_{0ms ( 0.00%)}	_226ms	_232ms
_{Req 4 - navto}	_{168ms (± 0.72%)}	_{170ms (± 0.76%)}	_{+1ms (+ 0.71%)}	_168ms	_173ms
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_{0 ( 0.00%)}	_1,356	_1,356
_{Req 5 - completionInfo}	_{56ms (± 1.23%)}	_{55ms (± 2.34%)}	_{-0ms (- 0.18%)}	_54ms	_58ms
_{CompilerTSServer - node (v14.15.1, x64)}
_{Req 1 - updateOpen}	_{1,521ms (± 0.68%)}	_{1,515ms (± 0.60%)}	_{-6ms (- 0.39%)}	_1,499ms	_1,542ms
_{Req 2 - geterr}	_{2,299ms (± 0.50%)}	_{2,462ms (± 0.41%)}	_{+164ms (+ 7.11%)}	_2,439ms	_2,486ms
_{Req 3 - references}	_{242ms (± 0.59%)}	_{242ms (± 0.51%)}	_{+0ms (+ 0.08%)}	_240ms	_246ms
_{Req 4 - navto}	_{178ms (± 0.49%)}	_{178ms (± 0.50%)}	_{-1ms (- 0.28%)}	_176ms	_179ms
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_{0 ( 0.00%)}	_1,518	_1,518
_{Req 5 - completionInfo}	_{55ms (± 1.06%)}	_{54ms (± 0.82%)}	_{-1ms (- 0.91%)}	_53ms	_55ms
_{xstateTSServer - node (v14.15.1, x64)}
_{Req 1 - updateOpen}	_{2,139ms (± 0.57%)}	_{2,128ms (± 0.54%)}	_{-11ms (- 0.52%)}	_2,096ms	_2,151ms
_{Req 2 - geterr}	_{772ms (± 0.58%)}	_{812ms (± 0.45%)}	_{+40ms (+ 5.14%)}	_806ms	_823ms
_{Req 3 - references}	_{67ms (± 2.13%)}	_{66ms (± 2.61%)}	_{-0ms (- 0.30%)}	_64ms	_71ms
_{Req 4 - navto}	_{228ms (± 0.53%)}	_{229ms (± 0.82%)}	_{+1ms (+ 0.57%)}	_225ms	_234ms
_{Req 5 - completionInfo count}	_{3,301 (± 0.00%)}	_{3,301 (± 0.00%)}	_{0 ( 0.00%)}	_3,301	_3,301
_{Req 5 - completionInfo}	_{276ms (± 0.62%)}	_{278ms (± 0.83%)}	_{+2ms (+ 0.65%)}	_274ms	_284ms

System

_{Machine Name}	_ts-ci-ubuntu
_Platform	_{linux 4.4.0-210-generic}
_Architecture	_x64
_{Available Memory}	_{16 GB}
_{Available Memory}	_{15 GB}
_CPUs	_{4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz}

Hosts

node (v10.16.3, x64)
node (v12.1.0, x64)
node (v14.15.1, x64)

Scenarios

Compiler-UnionsTSServer - node (v10.16.3, x64)
Compiler-UnionsTSServer - node (v12.1.0, x64)
Compiler-UnionsTSServer - node (v14.15.1, x64)
CompilerTSServer - node (v10.16.3, x64)
CompilerTSServer - node (v12.1.0, x64)
CompilerTSServer - node (v14.15.1, x64)
xstateTSServer - node (v10.16.3, x64)
xstateTSServer - node (v12.1.0, x64)
xstateTSServer - node (v14.15.1, x64)

_Benchmark	_Name	_Iterations
_Current	₄₉₈₈₆	₁₀
_Baseline	_main	₁₀

Developer Information:

Download Benchmark

Aug 31 '22 19:08 typescript-bot

Looks like some pretty dramatic performance overhead.

Aug 31 '22 21:08 ahejlsberg

@typescript-bot user test this inline @typescript-bot test this

Looks like some pretty dramatic performance overhead.

Some of the union construction work to support both kinds of null/undefined simultaneously may be more costly than I thought, I'll have to see where the time is going.

Aug 31 '22 23:08 weswigham

Heya @weswigham, I've started to run the diff-based user code test suite on this PR at b18eca0f207d65f60338e403170e1cdfd5453633. You can monitor the build here.

Update: The results are in!

Aug 31 '22 23:08 typescript-bot

Heya @weswigham, I've started to run the extended test suite on this PR at b18eca0f207d65f60338e403170e1cdfd5453633. You can monitor the build here.

Aug 31 '22 23:08 typescript-bot

@weswigham Here are the results of running the user test suite comparing main and refs/pull/49886/merge:

Something interesting changed - please have a look.

Details

webpack

tsconfig.json

[NEW] error TS2322: Type '{ timestamp: boolean; hash: boolean; } | { timestamp: boolean; hash?: undefined; }' is not assignable to type '{ hash?: boolean; timestamp?: boolean; }'.
- lib/config/defaults.js#L444
- lib/config/defaults.js#L447
[NEW] error TS2769: No overload matches this call.
- lib/dependencies/HarmonyExportInitFragment.js#L71
- lib/dependencies/HarmonyExportInitFragment.js#L85
[NEW] error TS2322: Type 'undefined' is not assignable to type '(value: any) => void'.
- lib/serialization/FileMiddleware.js#L587
[NEW] error TS2322: Type 'undefined' is not assignable to type '(reason?: any) => void'.
- lib/serialization/FileMiddleware.js#L588

tsconfig.types.json

[NEW] error TS2322: Type 'undefined' is not assignable to type 'Pick<ArgumentConfig, "values" | "type">'.
- lib/cli.js#L167
[NEW] error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Record<"filter" | "sort" | "result" | "extract" | "filterSorted" | "groupResults" | "sortResults" | "filterResults" | "merge" | "getItemName" | "getItemFactory", Map<string, SyncBailHook<[...], any, UnsetAdditionalOptions>[]>>'.
- lib/stats/StatsFactory.js#L77
- lib/stats/StatsFactory.js#L144
[MISSING] error TS2322: Type 'undefined' is not assignable to type 'Pick<ArgumentConfig, "type" | "values">'.
- lib/cli.js#L167
[MISSING] error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Record<"sort" | "filter" | "result" | "extract" | "filterSorted" | "groupResults" | "sortResults" | "filterResults" | "merge" | "getItemName" | "getItemFactory", Map<string, SyncBailHook<[...], any, UnsetAdditionalOptions>[]>>'.
- lib/stats/StatsFactory.js#L77
- lib/stats/StatsFactory.js#L144

Sep 01 '22 00:09 typescript-bot

Heya @weswigham, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

Sep 01 '22 00:09 typescript-bot

@typescript-bot user test this inline @typescript-bot test this

again - I think the merge conflict (I'm going to keep getting so many of those on this PR) was preventing the merge ref from updating, since some of the failures reported I already fixed locally...

Sep 01 '22 00:09 weswigham

Heya @weswigham, I'm starting to run the diff-based user code test suite on this PR at 5173a07ba7a5e5eb3f09049f0d9aa69c7b72330b. Hold tight - I'll update this comment with the log link once the build has been queued.

Update: The results are in!

Sep 01 '22 00:09 typescript-bot

Heya @weswigham, I'm starting to run the extended test suite on this PR at 5173a07ba7a5e5eb3f09049f0d9aa69c7b72330b. Hold tight - I'll update this comment with the log link once the build has been queued.

Sep 01 '22 00:09 typescript-bot

@weswigham Here are the results of running the user test suite comparing main and refs/pull/49886/merge:

Something interesting changed - please have a look.

Details

webpack

tsconfig.json

[NEW] error TS2769: No overload matches this call.
- lib/dependencies/HarmonyExportInitFragment.js#L71
- lib/dependencies/HarmonyExportInitFragment.js#L85
[NEW] error TS2322: Type 'undefined' is not assignable to type '(value: any) => void'.
- lib/serialization/FileMiddleware.js#L587
[NEW] error TS2322: Type 'undefined' is not assignable to type '(reason?: any) => void'.
- lib/serialization/FileMiddleware.js#L588

tsconfig.types.json

[NEW] error TS2322: Type 'undefined' is not assignable to type 'Pick<ArgumentConfig, "values" | "type">'.
- lib/cli.js#L167
[NEW] error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Record<"filter" | "sort" | "result" | "extract" | "filterSorted" | "groupResults" | "sortResults" | "filterResults" | "merge" | "getItemName" | "getItemFactory", Map<string, SyncBailHook<[...], any, UnsetAdditionalOptions>[]>>'.
- lib/stats/StatsFactory.js#L77
- lib/stats/StatsFactory.js#L144
[MISSING] error TS2322: Type 'undefined' is not assignable to type 'Pick<ArgumentConfig, "type" | "values">'.
- lib/cli.js#L167
[MISSING] error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'Record<"sort" | "filter" | "result" | "extract" | "filterSorted" | "groupResults" | "sortResults" | "filterResults" | "merge" | "getItemName" | "getItemFactory", Map<string, SyncBailHook<[...], any, UnsetAdditionalOptions>[]>>'.
- lib/stats/StatsFactory.js#L77
- lib/stats/StatsFactory.js#L144

Sep 01 '22 00:09 typescript-bot

Heya @weswigham, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

Sep 01 '22 00:09 typescript-bot