microsoft
Huge jump in tsc memory usage from 5.5.4 -> 5.6.2
🔎 Search Terms
"transformTime time", "compile time"
🕗 Version & Regression Information
- This changed between versions 5.5.4 and 5.6.2
⏯ Playground Link
No response
💻 Code
tsc on 5.6.2 is taking up significantly more memory for unknown reasons. No other dep changes other than typescript upgrade. I've looked into various articles / guides to try and track down how to further investigate this, but thus far I have not found a way to really track what is newly taking up so much more memory.
🙁 Actual behavior
After upgrading to 5.6, the memory usage of my tsc is more than double with no other changes to my .tsconfig.json file or the underlying code
🙂 Expected behavior
Before upgrade to 5.6, when running tsc on my project I get the following diagnostics.
Additional information about the issue
No response
Very difficult to speculate on what this might be. https://github.com/microsoft/TypeScript/wiki/Performance has information on how to run profiling, or you can share a repro with us
This tsc run is in large private monorepo that I won't be able to share access to. Is there anything in particular from the performance page that I might be able to run that would be specifically beneficial in tracking down the increased memory usage? Anything I might be able to pass of to the team?
The best tool for figuring out what's taking up more memory would be a heap snapshot, but a heap snapshot would also contain all of your code, so you'd have to diagnose it yourself.
The two runs don't quite look apples-to-apples - are you sure you're doing a clean build in both cases? I'm not sure how the first one reports 13s of transform time on a 12s execution (????)
Just to double check, I re ran these tsc runs just now. Only different in these two runs in the typescript version.
Typescript 5.5.4
Typescript 5.6.2
I'll go down the route and see I cant get some heap snapshots and see if anything looks suspicious.
I also guess my broader concern here is that just the TS version bump caused such a massive jump in memory used. Seems odd that there is something in our stack that is specifically causing that. I would have thought this type of thing would have come up for many others in the upgrade.
One thing you may consider is bisecting this with https://www.npmjs.com/package/every-ts, if the memory jump is clear-cut and not spread out over the entire dev cycle.
Maybe I'm misunderstanding, but in my case there isn't anything to bisect? Quite literally the only change from the two tsc runs above is changing the typescript version.
every-ts builds TS from source, so it bisects the repo and all of the changes we made between two versions.
Okay, so while I couldn't get every-ts to work. I just brute forced it and did a manual bisect. Here are some interesting numbers and the exact versions where there was a sizable memory jump. For the sake being forward looking, I also tested the latest 5.7 version, which actually saw a big memory drop, so it seems like maybe someone on the team found something already?
| Version | Mem Used |
|---|---|
| 5.7.0-dev.20241008 | 2719119K |
| 5.6.3 | 5550198K |
| 5.6.0-dev.20240817 | 5546372K |
| 5.6.0-dev.20240629 | 3625724K |
| 5.5.4 | 2047049K |
I'd be interested in what's in your .data/TypeScript dir (if you ls it), but it is moderately off-topic. If you delete .data and to do an every-ts fetch, it may resolve itself.
After deleting the .data dir and just running every-ts I get the follow.
Hm, I can only think that your version of git is out of date, or that you have a corporate proxy which is screwing with GitHub pulls. I can only suggest updating git or deleting and retrying. I've seen that error before, but not in a while.
I can maybe look into that. But is that necessary given the information I provide above with manually bisecting the releases?
The table seems to only narrow it down to a month and a half, which is half of our three month dev cycle, so unfortunately doesn't say much.
If you'd like to go farther, you could manually build from source, then run npm ci && npx hereby tsc && built/local/tsc.js -p path/to/tsconfig.json while bisecting.
Otherwise if you think it's fixed in 5.7, then that's fine, in which case this could be closed.
I think maybe my table wasn't clear. 06/28 -> 06/29 is where the first major jump in memory usage happened, than the next big jump being on 08/16 -> 08/17
Given that in the 5.6 dev cycle there were major shifts in memory usage of TSC, I wouldn't say that it's "fixed" in 5.7 until there is a RC out, which I can definitely test when we get close to that. But for 5.6, there was a major regression in memory usage that feels like it's maybe worth addressing on the current version?
Ah, I see.
5.6.0-dev.20240628to5.6.0-dev.20240629is 1b867c52b2a54cff51125bf915b5c8790334a376...cb98634f4915bbf7468d33a380a54a372fb94e265.6.0-dev.20240816to5.6.0-dev.20240817is 856e472104bc8f000d936642b2f46ece260ffb96...d06fb82d7c79cf26893b50d7787af0fe2ffbe8455.7.0-dev.20241007to5.7.0-dev.20241008is 44331b9fa32999d4aade7433c10cd3ec8596957e...cd6c0a0b6b8659f49344951219c74a516fdec9a2
Nothing here jumps out at me.
May be its "noEmit" now reports declaration errors so calculates those errors. which didnt happen before and hence the extra time and memory spent in that?
To be super clear... I didnt test 5.7.0-dev.20241007 to 5.7.0-dev.20241008 to see when the memory dropped again. I can try to track down when the memory dropped. I was only hunting for the increases in memeory.
Would it be fair to relabel this away from "Needs more info" at this point?
I don't think we have enough information to fix the bug. Even knowing a specific commit isn't really going to help since we don't have a way to analyze the root cause.
Also noticing increased memory usage leading to heap out of memory panic when running type checking during application builds (using nextjs). Originally commented on an open issue nextjs. But after downgrading typescript to 5.5.4 (from 5.8.2) the out of memory error was resolved.
Running tsc with --extendedDiagnostics gives:
Typescript 5.5.4:
Files: 4515
Lines of Library: 38324
Lines of Definitions: 400938
Lines of TypeScript: 123511
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 667476
Symbols: 1338789
Types: 471274
Instantiations: 4236717
Memory used: 1663318K
Assignability cache size: 268302
Identity cache size: 16385
Subtype cache size: 9336
Strict subtype cache size: 5904
Tracing time: 0.41s
I/O Read time: 0.92s
Parse time: 1.03s
ResolveTypeReference time: 0.03s
ResolveModule time: 1.12s
ResolveLibrary time: 0.01s
Program time: 3.49s
Bind time: 0.80s
Check time: 14.65s
printTime time: 0.00s
Emit time: 0.00s
Dump types time: 8.34s
Total time: 18.94s
Typescript 5.8.2:
Files: 4521
Lines of Library: 39534
Lines of Definitions: 401066
Lines of TypeScript: 123511
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 669367
Symbols: 1334751
Types: 468933
Instantiations: 4243341
Memory used: 1629155K
Assignability cache size: 268444
Identity cache size: 16287
Subtype cache size: 9077
Strict subtype cache size: 5835
Tracing time: 0.37s
I/O Read time: 0.95s
Parse time: 1.02s
ResolveTypeReference time: 0.02s
ResolveModule time: 0.90s
ResolveLibrary time: 0.01s
Program time: 3.39s
Bind time: 0.81s
Check time: 14.88s
printTime time: 0.00s
Emit time: 0.00s
Dump types time: 8.38s
Total time: 19.07s
So reporting lower marginally lower memory used in 5.8.2 - but still 5.8.2 is giving me the out of memory, whereas 5.5.4 is not, so the numbers don't quite add up.
https://www.npmjs.com/package/pprof-it can be used to take a memory allocation profile, but looking t the above, those two memory usage counts are not very far apart and could be attributed to basically anything. Finding a specific place between those two releases that changed things would be more helpful.
But, everyone's issues are different, it seems pretty unlikely that your issue is the same as this one unless you're using the same exact libraries and see the same bump at the same versions. 5.5 to 5.8 is far too large of a range to debug.
Hi all, I was able to recreate a simple reproduction of the issue and shared it here https://github.com/microsoft/TypeScript/issues/62660
At the core, it seems to stem from a difference in how the two versions of typescript handle certain kinds of memory allocation issues with large types that sometimes cause The inferred type of this node exceeds the maximum length the compiler will serialize. An explicit type annotation is needed.
