nx icon indicating copy to clipboard operation
nx copied to clipboard

perf(core): optimize project graph builder and operators

Open adwait1290 opened this issue 2 weeks ago • 3 comments

Current Behavior

removeDependenciesWithNode (project-graph-builder.ts:354-372)

When removing a node, the current implementation has O(n²) complexity with expensive array operations:

For each source's dependencies:    <- O(n) sources
  For each dependency (by index):  <- O(n) deps per source
    If target matches:
      Create new array via slice()  <- O(n) per match!

Problem: The nested loop plus array slice operations create significant overhead when removing nodes from large graphs.

withDeps (operators.ts:83-132)

The recursive graph traversal uses Array.indexOf() for visited tracking:

visitedNodes.indexOf(name) > -1  <- O(n) per recursive call!
visitedEdges.indexOf(name) > -1  <- O(n) per recursive call!

Problem: In recursive traversal, these O(n) lookups compound. For a graph depth of d with n nodes:

  • Worst case: O(n × d) lookups, each O(n) = O(n² × d)

Expected Behavior

removeDependenciesWithNode - O(n²) → O(n)

                    BEFORE                          AFTER
                    ═══════                         ═════
             ┌─────────────────────┐        ┌────────────────────┐
             │ For each source (n) │        │ For each source (n)│
             └──────────┬──────────┘        └──────────┬─────────┘
                        │                              │
             ┌──────────▼──────────┐        ┌──────────▼─────────┐
             │For each dep[i] (n)  │        │ Single filter()    │
             │  if target matches: │        │   remove all       │
             │    slice(0,i)  O(n) │        │   matching targets │
             │    slice(i+1)  O(n) │        │       O(n)         │
             └──────────┬──────────┘        └──────────┬─────────┘
                        │                              │
              Total: O(n²) + slice                Total: O(n)
                   overhead                    single pass filter

withDeps - O(n² × d) → O(n × d)

                    BEFORE                          AFTER
                    ═══════                         ═════
              ┌──────────────────┐          ┌──────────────────┐
              │ visitedNodes = []│          │visitedNodes = Set│
              │ visitedEdges = []│          │visitedEdges = Set│
              └────────┬─────────┘          └────────┬─────────┘
                       │                             │
              ┌────────▼─────────┐          ┌────────▼─────────┐
              │  recurNodes(n)   │          │  recurNodes(n)   │
              │                  │          │                  │
              │ indexOf() = O(n) │          │  has() = O(1)    │
              │ push()           │          │  add()           │
              └────────┬─────────┘          └────────┬─────────┘
                       │                             │
                       ▼                             ▼
              For depth d nodes:            For depth d nodes:
              O(n) × d = O(n×d)             O(1) × d = O(d)
              lookups each O(n)             lookups each O(1)
              
              Total: O(n² × d)              Total: O(n × d)

Performance Impact

Operation Before After Improvement
removeDependenciesWithNode O(n²) + slice overhead O(n) ~n× faster
withDeps (visited check) O(n) per call O(1) per call ~n× faster

Example: For a graph with 500 nodes, depth 10:

  • removeDependenciesWithNode: 250,000 ops → 500 ops
  • withDeps lookups: 2,500,000 ops → 5,000 ops

Changes

  1. project-graph-builder.ts:

    • Replace nested loop + slice with single-pass filter()
    • Only reassign when filter actually removes items
    • Maintains all existing behavior including empty array deletion
  2. operators.ts:

    • Convert visitedNodes array to Set<string>
    • Convert visitedEdges array to Set<string>
    • Replace .indexOf() > -1 with .has()
    • Replace .push() with .add()

Related Issue(s)

Contributes to #32265

Merge Dependencies

This PR has no dependencies and can be merged independently.

Must be merged BEFORE: #33748


adwait1290 avatar Dec 08 '25 04:12 adwait1290

Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
Latest commit 33086b445b0f378a3f5b02003aebec6e21bafa14

netlify[bot] avatar Dec 08 '25 04:12 netlify[bot]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nx-dev Ready Ready Preview Dec 8, 2025 4:58am

vercel[bot] avatar Dec 08 '25 04:12 vercel[bot]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@adwait1290 We are very grateful for your enthusiasm to contribute, I kindly request that you please stop sending these AI assisted micro-perf PRs now. In future, please open an issue regarding your plans and do not simply send pages worth of micro PRs without open communication.

Upon deeper inspection in some cases, we have found that they are not resulting in real-world performance wins, and instead create regressions because they are not considering memory and GC overhead of the whole system.

We will work on better benchmarking infrastructure on our side to have greater confidence in CI as to whether these kinds of PRs are actually net wins but for now each individual PR requires a thorough investigation by the team and you are sending far, far too many.

To reduce noise on the repo, I am going to close this, but rest assured it will be looked at as part of our performance optimization and benchmarking effort and merged in if it creates a provable net win.

Thank you once again for your keenness to help make Nx the best it can be, we really appreciate it!

JamesHenry avatar Dec 11 '25 10:12 JamesHenry

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

github-actions[bot] avatar Dec 17 '25 00:12 github-actions[bot]