microsoft
Performance regression with `--incremental` between `native-preview-0.20251211.1` and `0.20251209.1`
Hi,
Apologies for the absence of repro. Hopefully with the small time range it will still be helpful to pinpoint a potential regression.
We noticed today that tsgo was noticeably slower on our codebase. We found that the difference was :
- between versions
typescriptteam.native-preview-0.20251209.1(fast) andtypescriptteam.native-preview-0.20251211.1(slow) - only when using
--incremental, and only on first run (ie. no pre-existing.tsbuildinfo)
We reproduced multiple times, on multiple machines, but on a single codebase.
| Metric | 0.20251209.1 |
0.20251211.1 |
Notes |
|---|---|---|---|
| Check time (incremental, first run) | 6.642s | 19.916s | Large slowdown |
| CPU (incremental, first run) | 419% CPU | 186% CPU | Multi-threading issue? |
| Check time (incremental, next runs) | <0.01s | <0.01s | No difference |
| Check time (incremental = false) | ~6.5s | ~6.5s | No difference |
Detailed logs
time and extendedDiagnostics logs of first --incremental run, with each version.
(there was no meaningful differences/logs for other configurations)
# ✅ version 0.20251209.1 is fast
➜ rm -f .tsconfig.tsbuildinfo
➜ time /Users/gui/.vscode/extensions/typescriptteam.native-preview-0.20251209.1-darwin-arm64/lib/tsgo --noEmit --project tsconfig.json --incremental --extendedDiagnostics
Files: 9918
Lines: 1489421
Identifiers: 1767364
Symbols: 8496581
Types: 2938919
Instantiations: 10101417
Memory used: 5176707K
Memory allocs: 70927013
Config time: 0.094s
BuildInfo read time: 0.000s
Parse time: 0.718s
Bind time: 0.000s
Check time: 6.642s
Emit time: 0.386s
Changes compute time: 0.168s
Total time: 8.011s
time: 41.01s user 2.36s system 419% cpu 10.350 total
# ❌ version 0.20251211.1 is slow
➜ rm -f .tsconfig.tsbuildinfo
➜ time /Users/gui/.vscode/extensions/typescriptteam.native-preview-0.20251211.1-darwin-arm64/lib/tsgo --noEmit --project tsconfig.json --incremental --extendedDiagnostics
Files: 9918
Lines: 1489421
Identifiers: 1767364
Symbols: 8496581
Types: 2938919
Instantiations: 10101417
Memory used: 5173748K
Memory allocs: 71095193
Config time: 0.102s
BuildInfo read time: 0.000s
Parse time: 0.715s
Bind time: 0.001s
Check time: 19.916s
Emit time: 0.447s
Changes compute time: 0.269s
Total time: 21.456s
time: 43.04s user 2.15s system 186% cpu 24.246 total
There were many changes in between these two versions. Could you build from source and git bisect?
Sure, could you point me to which commits correspond to the two releases? I can't find tags/release or publish commits.
The npm package has that in the package.json, but I guess not the VS Code extension.
It'd be 8372401975e9186b2270c89c2c821e2efaf9c522 to ddf0b72580c2382c6cab6c2a98eeafb7d3818481
8372401975e9186b2270c89c2c821e2efaf9c522...ddf0b72580c2382c6cab6c2a98eeafb7d3818481
@jakebailey I'm sorry to tell you that git bisect pointed to #2314 😕 . I couldn't check the individual PR commits, the branch has been deleted.
Didn't anyone else or any test suite notice the issue? It's a 3x slowdown here, not subtle. I still don't see how I could make a repro, but let me know if I can help further.
Didn't anyone else or any test suite notice the issue? It's a 3x slowdown here, not subtle.
There is no test that can catch this sort of issue. (There also was no way in the old compiler either...)
I couldn't check the individual PR commits, the branch has been deleted.
It wouldn't have been worth it, so no worries there.
I still don't see how I could make a repro, but let me know if I can help further.
Since you can show the issue, can you set --pprofDir /path/to/some/dir before/after, and upload the profiles? I'd just want the runs you showed above where one was obviously longer than the other.
Best of all would be to have access to the code where this is happening, of course.
Didn't anyone else or any test suite notice the issue? It's a 3x slowdown here, not subtle.
There is no test that can catch this sort of issue. (There also was no way in the old compiler either...)
I'd imagine possible to :
- snapshot 5-10 big open source repos that tsgo supports well.
- have some CI action that build them all with different flags and notify if performance is notably degraded.
- it could also help to spot regressions in "real life" errors that may have been missed by tests suites.
Since you can show the issue, can you set
--pprofDir /path/to/some/dirbefore/after, and upload the profiles? I'd just want the runs you showed above where one was obviously longer than the other.Best of all would be to have access to the code where this is happening, of course.
Ok, I think I got something better than sharing my profiles.
I picked one of our dependency with ~big types (https://github.com/mikro-orm/mikro-orm) and it reproduces with a x1.7 slowdown.
With tsgo-GOOD, built before #2314:
With tsgo-BAD, built after #2314:
I suspect all projects are affected, so this issue could actually be important.
You can probably reproduce on another big enough project you are most familiar with building instead of Mikro-ORM.
If it helps, here is the setup to quickly repro on Mikro-ORM:
git clone [email protected]:mikro-orm/mikro-orm.git
cd mikro-orm
npm install
# Unfortunately, MikroORM tsconfigs need minimal patching for tsgo:
sed -i '' 's|"src|"./src|g' packages/core/tsconfig.json # Non-relative paths are not allowed
sed -i '' 's|"packages|"./packages|g' tsconfig.json # Non-relative paths are not allowed
sed -i '' '/"baseUrl"\s*:/d' tsconfig.json # Option 'baseUrl' has been removed.
# I pre-built tsgo-GOOD (before #2314) and tsgo-BAD (after #2314)
rm -f ./packages/core/tsconfig.tsbuildinfo && time /Users/gui/Dev/typescript-go/built/local/tsgo-GOOD --noEmit --project packages/core/tsconfig.json --incremental --extendedDiagnostics
rm -f ./packages/core/tsconfig.tsbuildinfo && time /Users/gui/Dev/typescript-go/built/local/tsgo-BAD --noEmit --project packages/core/tsconfig.json --incremental --extendedDiagnostics
Reminder: the issue is only with --incremental and on first run (no pre-existing .tsbuildinfo)
Same here, b0fed9f5bb0d29327e5ee8ba8e0051592cd76639 resulted in 2× slowdown in the codebase I'm working on.
% go version
go version go1.25.5 X:nodwarf5 linux/amd64
% uname -sr
Linux 6.18.1-arch1-2
e93d36401a3f6c19a5e7384abbbf40aa210d7cda:
% git checkout e93d36401a3f6c19a5e7384abbbf40aa210d7cda
% go build ./cmd/tsgo
% rm node_modules/.ts-build-info
% ~/typescript-go/tsgo --extendedDiagnostics
Files: 5857
Lines: 507475
Identifiers: 485501
Symbols: 1945774
Types: 1860168
Instantiations: 4227816
Memory used: 3393787K
Memory allocs: 48136781
Config time: 0.068s
BuildInfo read time: 0.000s
Parse time: 0.151s
Bind time: 0.000s
Check time: 6.835s
Emit time: 0.069s
Changes compute time: 0.087s
Total time: 7.215s
b0fed9f5bb0d29327e5ee8ba8e0051592cd76639:
% git checkout b0fed9f5bb0d29327e5ee8ba8e0051592cd76639
% go build ./cmd/tsgo
% rm node_modules/.ts-build-info
% ~/typescript-go/tsgo --extendedDiagnostics
Files: 5857
Lines: 507475
Identifiers: 485501
Symbols: 1945774
Types: 1860168
Instantiations: 4227816
Memory used: 3392827K
Memory allocs: 48096576
Config time: 0.069s
BuildInfo read time: 0.000s
Parse time: 0.134s
Bind time: 0.000s
Check time: 15.857s
Emit time: 0.071s
Changes compute time: 0.107s
Total time: 16.241s
And if incremenetal is disabled, it's fast again:
Files: 5857
Lines: 507475
Identifiers: 485501
Symbols: 1946002
Types: 1860168
Instantiations: 4227816
Memory used: 3364410K
Memory allocs: 46842984
Config time: 0.069s
Parse time: 0.140s
Bind time: 0.016s
Check time: 6.803s
Emit time: 0.001s
Total time: 7.033s
tsconfig.json:
{
"compilerOptions": {
"paths": {
"@/*": ["./src/*"],
"*": ["./*"]
},
"tsBuildInfoFile": "./node_modules/.ts-build-info",
"incremental": true,
"types": ["vitest/globals", "vite/client", "@types/node"],
"experimentalDecorators": true,
"target": "ESNext",
"lib": ["DOM", "DOM.Iterable", "ESNext", "ESNext.AsyncIterable"],
"allowJs": false,
"allowImportingTsExtensions": true,
"skipLibCheck": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"forceConsistentCasingInFileNames": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"noFallthroughCasesInSwitch": true,
"noUncheckedSideEffectImports": true,
"noImplicitReturns": true,
"noUncheckedIndexedAccess": false,
"useDefineForClassFields": true,
"moduleDetection": "force"
},
"include": ["src", "scripts"],
"watchOptions": {
"excludeDirectories": ["node_modules"]
}
}
I've attached profiles for all runs, please tell if there's something else I can do except for sharing code (which does not belong to me). I'll try to extract a repro, but no promises, it's a pretty convoluted project.
