zod
zod copied to clipboard
colinhacks
TS compilation perf: steal faster extendShape type from tRPC
I have some empirical evidence in a decently sized closed-source project that liberal use of ZodObject.extend() seems to degrade TS compilation and intellisense performance quite rapidly. I remembered seeing an equivalent implementation of extendShape in tRPC (in that project, called Overwrite) that looked simpler. I did some light profiling on it using said closed-source project and here are the results:
// Zod@jussisaurio
% npx tsc --noEmit --extendedDiagnostics --incremental false
Files: 1248
Lines of Library: 39145
Lines of Definitions: 126023
Lines of TypeScript: 40594
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 183660
Symbols: 290202
Types: 147133
Instantiations: 2361427
Memory used: 345043K
Assignability cache size: 34109
Identity cache size: 1470
Subtype cache size: 708
Strict subtype cache size: 22394
I/O Read time: 0.04s
Parse time: 0.38s
ResolveModule time: 0.09s
ResolveTypeReference time: 0.01s
ResolveLibrary time: 0.01s
Program time: 0.60s
Bind time: 0.17s
Check time: 1.73s
printTime time: 0.00s
Emit time: 0.00s
Total time: 2.49s
// [email protected]
% npx tsc --noEmit --extendedDiagnostics --incremental false
Files: 1248
Lines of Library: 39145
Lines of Definitions: 126004
Lines of TypeScript: 40594
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 183556
Symbols: 286851
Types: 152232
Instantiations: 7692583
Memory used: 356978K
Assignability cache size: 33682
Identity cache size: 1464
Subtype cache size: 708
Strict subtype cache size: 22394
I/O Read time: 0.04s
Parse time: 0.39s
ResolveModule time: 0.10s
ResolveTypeReference time: 0.01s
ResolveLibrary time: 0.01s
Program time: 0.60s
Bind time: 0.17s
Check time: 2.80s
printTime time: 0.00s
Emit time: 0.00s
Total time: 3.57s
// Zod@jussisaurio, run #2
% npx tsc --noEmit --extendedDiagnostics --incremental false
Files: 1248
Lines of Library: 39145
Lines of Definitions: 126023
Lines of TypeScript: 40594
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 183660
Symbols: 290202
Types: 147133
Instantiations: 2361427
Memory used: 345221K
Assignability cache size: 34109
Identity cache size: 1470
Subtype cache size: 708
Strict subtype cache size: 22394
I/O Read time: 0.04s
Parse time: 0.40s
ResolveModule time: 0.10s
ResolveTypeReference time: 0.01s
ResolveLibrary time: 0.01s
Program time: 0.63s
Bind time: 0.17s
Check time: 1.80s
printTime time: 0.00s
Emit time: 0.00s
Total time: 2.60s
// [email protected], run #2
% npx tsc --noEmit --extendedDiagnostics --incremental false
Files: 1248
Lines of Library: 39145
Lines of Definitions: 126004
Lines of TypeScript: 40594
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 183556
Symbols: 286851
Types: 152232
Instantiations: 7692583
Memory used: 356570K
Assignability cache size: 33682
Identity cache size: 1464
Subtype cache size: 708
Strict subtype cache size: 22394
I/O Read time: 0.04s
Parse time: 0.40s
ResolveModule time: 0.10s
ResolveTypeReference time: 0.01s
ResolveLibrary time: 0.01s
Program time: 0.63s
Bind time: 0.16s
Check time: 2.81s
printTime time: 0.00s
Emit time: 0.00s
Total time: 3.61s
Main thing (apart from faster runtime) is the dramatically lower number of type instantiations.
Deploy Preview for guileless-rolypoly-866f8a ready!
Built without sensitive environment variables
| Name | Link |
|---|---|
| Latest commit | 5b2ad4c95321e8441619a217f71b1e8b313be7f8 |
| Latest deploy log | https://app.netlify.com/sites/guileless-rolypoly-866f8a/deploys/651d9277c512c00008157d30 |
| Deploy Preview | https://deploy-preview-2839--guileless-rolypoly-866f8a.netlify.app |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify site configuration.
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
I feel like there are edge cases where dropping flatten breaks the inferred type signature...but I can't find them.
I'll do a little more testing then merge. Thanks!
FWIW I can try to run some perf tests on how much keeping flatten has an impact. I'm guessing though that replacing identity with something like & {} in the definition of flatten would also make it perform better due to less instantiations
Used this: https://github.com/jussisaurio/zod-ts-perftest (there are already pregenerated zod schemas in the repo)
Running npm run build-bench, three runs each:
master: 4.51s, 4.56s, 4.51s PR #2839 (this one): 3.69s, 3.61s, 3.72s PR #2839 + flatten added back: 3.77s, 3.77s, 3.72s PR #2845: 2.29s, 2.20s, 2.21s Both PRs: 2.14s, 2.14s, 2.16s
Not too much of a difference at all between keeping or removing flatten, but for some reason this benchmark is really sensitive to the changes in #2845 😲 I'm sure this particular benchmark is not very well rounded (zod-ts-perftest only generates objects, strings, numbers, booleans, plus a crapton of extends and omits).
We use extend quite a bit. This PR is a marked improvement.
zod 3.22.4:
npx tsc --noEmit --extendedDiagnostics --incremental false
Files: 3394
Lines of Library: 40241
Lines of Definitions: 310600
Lines of TypeScript: 73937
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 592152
Symbols: 950503
Types: 311682
Instantiations: 8492589
Memory used: 1151591K
Assignability cache size: 273407
Identity cache size: 23254
Subtype cache size: 5426
Strict subtype cache size: 7230
I/O Read time: 0.04s
Parse time: 0.65s
ResolveModule time: 0.20s
ResolveTypeReference time: 0.00s
ResolveLibrary time: 0.01s
Program time: 1.02s
Bind time: 0.34s
Check time: 7.75s
printTime time: 0.00s
Emit time: 0.00s
Total time: 9.12s
zod 3.22.4 with this patch:
npx tsc --noEmit --extendedDiagnostics --incremental false
Files: 3394
Lines of Library: 40241
Lines of Definitions: 310610
Lines of TypeScript: 73937
Lines of JavaScript: 0
Lines of JSON: 0
Lines of Other: 0
Identifiers: 592160
Symbols: 950391
Types: 310870
Instantiations: 6238717
Memory used: 1058367K
Assignability cache size: 273343
Identity cache size: 23254
Subtype cache size: 5426
Strict subtype cache size: 7230
I/O Read time: 0.04s
Parse time: 0.66s
ResolveModule time: 0.21s
ResolveTypeReference time: 0.00s
ResolveLibrary time: 0.01s
Program time: 1.03s
Bind time: 0.35s
Check time: 7.32s
printTime time: 0.00s
Emit time: 0.00s
Total time: 8.70s
The time to check the whole app isn't a big deal, but VSCode's TS intellisense is a pain. It slowed down substantially after I rewrote a bunch of schemas using extend. We have a number of nested extend schemas in big unions.
