nx icon indicating copy to clipboard operation
nx copied to clipboard

fix(core): optimize daemon server hot paths

Open adwait1290 opened this issue 2 weeks ago • 4 comments

Current Behavior

The daemon server has several performance inefficiencies in hot paths:

sync-generators.ts

O(n²) Conflict Detection (lines 322-331)

for (const result of initialResults) {
  if (conflictRunResults.every((r) => r.generatorName !== result.generatorName)) {
    results.push(result);
  }
}

For each of n results, .every() iterates m conflict results = O(n*m)

Redundant Set Creation (lines 113-119)

const uniqueSyncGenerators = new Set<string>([
  ...registeredSyncGenerators.globalGenerators,
  ...registeredSyncGenerators.taskGenerators,
]);
for (const generator of uniqueSyncGenerators) {
  scheduledGenerators.add(generator);
}

Creates intermediate Set only to iterate it once.

outputs-tracking.ts

Redundant dirname() calls (lines 19-26, 87-92)

while (current != dirname(current)) {  // call #1
  // ...
  current = dirname(current);          // call #2
}

dirname() is called twice per iteration.

Late disabled check

export async function recordOutputsHash(_outputs, hash) {
  const outputs = await normalizeOutputs(_outputs);  // expensive!
  if (disabled) return;  // too late, work already done
  _recordOutputsHash(outputs, hash);
}

Expected Behavior

Complexity Improvements

┌────────────────────────────────────────────────────────────────┐
│ sync-generators.ts - processConflictingGenerators              │
├────────────────────────────────────────────────────────────────┤
│ BEFORE: O(n*m) - For each result, check all conflict results   │
│                                                                │
│   results (n)     conflicts (m)    operations                  │
│   ─────────────   ─────────────    ──────────                  │
│      10      ×       5         =      50                       │
│      50      ×      20         =   1,000                       │
│     100      ×      50         =   5,000                       │
│                                                                │
│ AFTER: O(n) - Set creation O(m), then O(1) lookups             │
│                                                                │
│   results (n)     operations                                   │
│   ─────────────   ──────────                                   │
│      10           10 + 5 = 15                                  │
│      50           50 + 20 = 70                                 │
│     100           100 + 50 = 150                               │
│                                                                │
│ Improvement: ~33x for 100 results with 50 conflicts            │
└────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────┐
│ outputs-tracking.ts - dirname() optimization                   │
├────────────────────────────────────────────────────────────────┤
│ BEFORE: 2 dirname() calls per loop iteration                   │
│ AFTER:  1 dirname() call per loop iteration                    │
│                                                                │
│ For a path like /workspace/dist/apps/myapp/main.js             │
│ (depth = 6 directories)                                        │
│                                                                │
│   outputs     depth    before    after    saved                │
│   ─────────   ─────    ──────    ─────    ─────                │
│      100    ×   6   =   1,200      600      600                │
│      500    ×   6   =   6,000    3,000    3,000                │
│    1,000    ×   6   =  12,000    6,000    6,000                │
│                                                                │
│ 50% reduction in dirname() calls                               │
└────────────────────────────────────────────────────────────────┘

Early Exit Optimization

┌────────────────────────────────────────────────────────────────┐
│ outputs-tracking.ts - disabled check order                     │
├────────────────────────────────────────────────────────────────┤
│ BEFORE: normalize() called even when disabled                  │
│                                                                │
│   recordOutputsHash()                                          │
│   ├── normalizeOutputs()  ← expensive FFI call                 │
│   ├── if (disabled) return  ← too late!                        │
│   └── _recordOutputsHash()                                     │
│                                                                │
│ AFTER: check disabled first                                    │
│                                                                │
│   recordOutputsHash()                                          │
│   ├── if (disabled) return  ← early exit                       │
│   ├── normalizeOutputs()                                       │
│   └── _recordOutputsHash()                                     │
│                                                                │
│ When disabled: 0 work instead of full normalization            │
└────────────────────────────────────────────────────────────────┘

Changes Made

sync-generators.ts

  • Use Set for O(1) lookup in processConflictingGenerators instead of O(n) .every()
  • Remove intermediate Set creation in collectAndScheduleSyncGenerators

outputs-tracking.ts

  • Calculate dirname() once per iteration, reuse result
  • Move disabled check before expensive normalizeOutputs() call
  • Make normalizeOutputs synchronous (underlying getFilesForOutputs is sync FFI)
  • Use Date.now() instead of new Date().getTime()

Test Plan

  • [x] All daemon server tests pass (11 tests)
  • [x] pnpm jest packages/nx/src/daemon/server/sync-generators.spec.ts - 1 test passes
  • [x] pnpm jest packages/nx/src/daemon/server/outputs-tracking.spec.ts - 5 tests pass

Related Issue(s)

Contributes to #32265, #33263

Merge Dependencies

This PR has no dependencies and can be merged independently.

Must be merged BEFORE: #33748


adwait1290 avatar Dec 08 '25 04:12 adwait1290

Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
Latest commit 06a4ddad2d7a74163feed43eb2960b80f696bb54

netlify[bot] avatar Dec 08 '25 04:12 netlify[bot]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nx-dev Ready Ready Preview Dec 11, 2025 7:24am

vercel[bot] avatar Dec 08 '25 04:12 vercel[bot]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

View your CI Pipeline Execution ↗ for commit 5edc9bc3acf2b8424b29dfb0a506b2da00fa375b

Command Status Duration Result
nx affected --targets=lint,test,test-kt,build,e... ❌ Failed 17m 52s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 2m 54s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 12s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 2s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2025-12-08 23:13:42 UTC

nx-cloud[bot] avatar Dec 08 '25 21:12 nx-cloud[bot]