nx
nx copied to clipboard
nrwl
perf(core): optimize project graph cache processing and workspace hash
Current Behavior
1. processProjectNode Uses Manual Object Construction
// packages/nx/src/project-graph/nx-deps-cache.ts (lines 414-417)
const fileDataFromCache = {} as any;
for (let f of cachedFileMap[projectName]) {
fileDataFromCache[f.file] = f;
}
Problem Visualization:
┌─────────────────────────────────────────────────────────────────┐
│ CURRENT: Manual Object Construction │
├─────────────────────────────────────────────────────────────────┤
│ │
│ cachedFileMap[project] = [ │
│ { file: "src/a.ts", hash: "abc" }, │
│ { file: "src/b.ts", hash: "def" }, │
│ { file: "src/c.ts", hash: "ghi" }, │
│ ...100 more files │
│ ] │
│ │
│ Step 1: Create empty object {} │
│ Step 2: Loop through all files (103 iterations) │
│ Step 3: Assign each file to object property │
│ │
│ Issues: │
│ - Using `as any` loses type safety │
│ - Manual loop is less idiomatic than Map constructor │
│ - Object property access is O(1) but creation is verbose │
│ │
└─────────────────────────────────────────────────────────────────┘
2. computeWorkspaceConfigHash Calls JSON.stringify Per Project
// packages/nx/src/daemon/server/project-graph-incremental-recomputation.ts (lines 212-224)
function computeWorkspaceConfigHash(projectsConfigurations) {
const projectConfigurationStrings = Object.entries(projectsConfigurations)
.sort(([projectNameA], [projectNameB]) =>
projectNameA.localeCompare(projectNameB)
)
.map(
([projectName, projectConfig]) =>
`${projectName}:${JSON.stringify(projectConfig)}` // EXPENSIVE
);
return hashArray(projectConfigurationStrings);
}
Cost Analysis:
┌─────────────────────────────────────────────────────────────────┐
│ JSON.stringify vs hashObject Performance │
├─────────────────────────────────────────────────────────────────┤
│ │
│ JSON.stringify(projectConfig): │
│ ├── Traverses entire object tree │
│ ├── Converts ALL values to JSON strings │
│ ├── Handles circular reference detection │
│ └── Creates intermediate string for EVERY value │
│ │
│ hashObject(projectConfig): │
│ ├── Sorts keys once │
│ ├── Primitives: direct string conversion (no stringify) │
│ │ - string → use directly │
│ │ - number → String(n) │
│ │ - boolean → String(b) │
│ ├── Objects/arrays: JSON.stringify (only when necessary) │
│ └── Passes to native xxhash (Rust, very fast) │
│ │
│ ProjectConfiguration typical breakdown: │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ Field │ Type │ JSON.stringify needed? │ │
│ ├────────────────────┼───────────┼────────────────────────┤ │
│ │ name │ string │ No (hashObject skips) │ │
│ │ root │ string │ No (hashObject skips) │ │
│ │ sourceRoot │ string │ No (hashObject skips) │ │
│ │ projectType │ string │ No (hashObject skips) │ │
│ │ targets │ object │ Yes (complex) │ │
│ │ tags │ string[] │ Yes (array) │ │
│ │ implicitDeps │ string[] │ Yes (array) │ │
│ └────────────────────┴───────────┴────────────────────────┘ │
│ │
│ ~60% of fields are primitives → 60% fewer stringify calls │
│ │
└─────────────────────────────────────────────────────────────────┘
When This Matters:
┌─────────────────────────────────────────────────────────────────┐
│ computeWorkspaceConfigHash Call Frequency │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Called from: processCollectedUpdatedAndDeletedFiles() │
│ Triggered by: ANY file change detected by daemon watcher │
│ │
│ Typical developer workflow: │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Action │ Hash computations │ │
│ ├─────────────────────────────┼───────────────────────────┤ │
│ │ Save a file │ 1 full workspace hash │ │
│ │ Git checkout branch │ N files → N hash calls │ │
│ │ npm install │ Many files → many calls │ │
│ │ IDE auto-save (frequent) │ 1 per save │ │
│ └─────────────────────────────┴───────────────────────────┘ │
│ │
│ For 200 project monorepo: │
│ - Each hash call: 200 × JSON.stringify │
│ - With hashObject: 200 × (partial stringify + native hash) │
│ │
└─────────────────────────────────────────────────────────────────┘
Expected Behavior
1. Clean Map Construction
┌─────────────────────────────────────────────────────────────────┐
│ NEW: Map Constructor Pattern │
├─────────────────────────────────────────────────────────────────┤
│ │
│ const fileDataFromCache = new Map( │
│ cachedFileMap[projectName].map((f) => [f.file, f]) │
│ ); │
│ │
│ Benefits: │
│ ✓ Single expression (more readable) │
│ ✓ Proper TypeScript typing (no `as any`) │
│ ✓ Map.get() for O(1) lookups │
│ ✓ Consistent with processNonProjectFiles (same pattern) │
│ │
│ Performance: Same O(n) complexity, cleaner code │
│ │
└─────────────────────────────────────────────────────────────────┘
2. Optimized Hash Computation
┌─────────────────────────────────────────────────────────────────┐
│ NEW: hashObject Integration │
├─────────────────────────────────────────────────────────────────┤
│ │
│ function computeWorkspaceConfigHash(projectsConfigurations) { │
│ const sortedNames = Object.keys(projectsConfigurations) │
│ .sort(); │
│ const projectConfigurationStrings = sortedNames.map( │
│ (projectName) => │
│ `${projectName}:${hashObject(projectsConfigurations[ │
│ projectName])}` │
│ ); │
│ return hashArray(projectConfigurationStrings); │
│ } │
│ │
│ Flow: │
│ 1. Sort project names once (same as before) │
│ 2. For each project config: │
│ - hashObject sorts keys │
│ - Primitives: direct string (no JSON overhead) │
│ - Objects: JSON.stringify (only complex values) │
│ - Pass to native xxhash │
│ 3. Hash the array of hashes │
│ │
│ Result: Same deterministic hash, fewer stringify operations │
│ │
└─────────────────────────────────────────────────────────────────┘
Why Accept This PR?
1. Consistency: Matches existing patterns in the codebase
┌─────────────────────────────────────────────────────────────────┐
│ Pattern Consistency │
├─────────────────────────────────────────────────────────────────┤
│ │
│ processNonProjectFiles (line 391): │
│ const cachedHashMap = new Map( │
│ cachedFiles.map((f) => [f.file, f]) │
│ ); │
│ │
│ processProjectNode (NOW): │
│ const fileDataFromCache = new Map( │
│ cachedFileMap[projectName].map((f) => [f.file, f]) │
│ ); │
│ │
│ ✓ Same pattern in both functions │
│ ✓ Easier to understand and maintain │
│ ✓ Removes type assertion hack (`as any`) │
│ │
└─────────────────────────────────────────────────────────────────┘
2. Performance: Leverages existing optimizations
┌─────────────────────────────────────────────────────────────────┐
│ hashObject Optimization (from previous PR) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ // hashObject now skips JSON.stringify for primitives: │
│ if (typeof value === 'string') parts.push(value); │
│ else if (typeof value === 'number') parts.push(String(value));│
│ else parts.push(JSON.stringify(value)); │
│ │
│ By using hashObject instead of JSON.stringify directly, │
│ computeWorkspaceConfigHash automatically benefits from │
│ the primitive optimization. │
│ │
│ Estimated savings per hash call: │
│ - 200 projects × 60% primitive fields = 120 fewer stringify │
│ - Per file save in daemon: 120 fewer operations │
│ │
└─────────────────────────────────────────────────────────────────┘
3. Safety: Better typing, fewer potential bugs
┌─────────────────────────────────────────────────────────────────┐
│ Type Safety Improvement │
├─────────────────────────────────────────────────────────────────┤
│ │
│ BEFORE: │
│ const fileDataFromCache = {} as any; // Type safety lost! │
│ fileDataFromCache[f.file] = f; // No type checking │
│ │
│ AFTER: │
│ const fileDataFromCache = new Map<string, FileData>(...) │
│ fileDataFromCache.get(f.file) // Typed as FileData │
│ │
│ Also: == changed to === for strict equality │
│ │
└─────────────────────────────────────────────────────────────────┘
Test Plan
- [x]
packages/nx/src/project-graph/nx-deps-cache.spec.ts- 17 tests pass - [x] No tests exist for project-graph-incremental-recomputation (daemon integration)
- [x] Hash output is deterministic (same inputs → same hash)
Related Issue(s)
Contributes to #32265
Merge Dependencies
Must be merged AFTER: #33739 Must be merged BEFORE: #33755
Deploy request for nx-docs pending review.
Visit the deploys page to approve it
| Name | Link |
|---|---|
| Latest commit | 9afdccee5eddfd7bf34d3e6e940ca5221ee3dff4 |
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
The latest updates on your projects. Learn more about Vercel for GitHub.
| Project | Deployment | Preview | Updated (UTC) |
|---|---|---|---|
| nx-dev |  Ready |
Preview | Dec 8, 2025 4:35am |
![vercel[bot] avatar](https://avatars.githubusercontent.com/in/8329?v=4 "Published by a pub.dev verified publisher"){kind=small}
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
![chatgpt-codex-connector[bot] avatar](https://avatars.githubusercontent.com/in/1144995?v=4 "Published by a pub.dev verified publisher"){kind=small}
@adwait1290 We are very grateful for your enthusiasm to contribute, I kindly request that you please stop sending these AI assisted micro-perf PRs now. In future, please open an issue regarding your plans and do not simply send pages worth of micro PRs without open communication.
Upon deeper inspection in some cases, we have found that they are not resulting in real-world performance wins, and instead create regressions because they are not considering memory and GC overhead of the whole system.
We will work on better benchmarking infrastructure on our side to have greater confidence in CI as to whether these kinds of PRs are actually net wins but for now each individual PR requires a thorough investigation by the team and you are sending far, far too many.
To reduce noise on the repo, I am going to close this, but rest assured it will be looked at as part of our performance optimization and benchmarking effort and merged in if it creates a provable net win.
Thank you once again for your keenness to help make Nx the best it can be, we really appreciate it!
This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}