msbuild icon indicating copy to clipboard operation
msbuild copied to clipboard

Refactor Node IDs for taskhost to use record struct

Open Copilot opened this issue 1 month ago • 5 comments

Fixes #12610

Context

Follow-up to #12577. The taskhost node identification used bit-packing to combine scheduledNodeId (8 bits) and HandshakeOptions (9 bits) into a single int, limiting multithreaded builds to 256 nodes. This approach was fragile and hard to maintain.

Changes Made

  • New TaskHostNodeKey record struct in CommunicationsUtilities.cs combining HandshakeOptions and NodeId as a clean, type-safe key
  • Updated NodeProviderOutOfProcTaskHost with optimized dictionary structure:
    • _nodeContexts keyed by TaskHostNodeKey for checking if a node with given (HandshakeOptions, scheduledNodeId) exists (acquisition logic)
    • _nodeIdToNodeKey for O(1) reverse lookup during node termination
    • _nodeIdToPacketFactory and _nodeIdToPacketHandler keyed by int for O(1) packet routing performance
    • _activeNodes tracks node IDs (int) for shutdown handling
    • Added atomic _nextNodeId counter for unique communication node ID generation
  • Simplified TaskHostTask - removed GenerateTaskHostNodeId method and associated bit-packing constants
  • Improved error handling - Added VerifyThrow assertions in DisconnectFromHost and CreateNode to verify context existence and prevent duplicate nodes
// Before: bit-packed int with 256-node limit
_taskHostNodeId = GenerateTaskHostNodeId(_scheduledNodeId, _requiredContext);
// (scheduledNodeId << 9) | ((int)handshakeOptions & 0x1FF)

// After: clean record struct, no limit
_taskHostNodeKey = new TaskHostNodeKey(_requiredContext, _scheduledNodeId);

This gives the best of both worlds:

  • Maintainability: Type-safe Keys for managing node lifecycle and acquisition
  • Performance: Direct Integer lookups for the high-frequency packet routing hot path

Testing

  • Added 7 unit tests for TaskHostNodeKey (equality, hash codes, dictionary keys, large node IDs)
  • All 86 existing TaskHost tests pass

Notes

  • Removes the NODE_ID_MAX_VALUE_FOR_MULTITHREADED (255) constraint entirely
  • SendData(int, INodePacket) throws NotImplementedException as it's required by INodeProvider interface but not used for task hosts (similar to TaskHostNodeManager)
Original prompt

This section details on the original issue you should resolve

<issue_title>[multithreaded] Refactor Node IDs for taskhost</issue_title> <issue_description>followup to dotnet/msbuild#12577 Refactor src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcTaskHost.cs , HandshakeOptions to use record struct as a key to nodes to increase maintanability and remove limitation to 256 nodes.</issue_description>

Comments on the Issue (you are @copilot in this section)

  • Fixes dotnet/msbuild#12610

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot avatar Nov 25 '25 14:11 Copilot

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

[!WARNING]

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • rcxvsblobprodcus328.vsblob.vsassets.io
    • Triggering command: /home/REDACTED/work/msbuild/msbuild/artifacts/bin/bootstrap/core/dotnet dotnet test src/Build.UnitTests/Microsoft.Build.Engine.UnitTests.csproj --filter FullyQualifiedName~TaskHost --no-build -v n (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot avatar Nov 25 '25 16:11 Copilot

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

[!WARNING]

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • rcxvsblobprodcus328.vsblob.vsassets.io
    • Triggering command: /home/REDACTED/work/msbuild/msbuild/artifacts/bin/bootstrap/core/dotnet dotnet test src/Build.UnitTests/Microsoft.Build.Engine.UnitTests.csproj --filter FullyQualifiedName~TaskHost --no-build -v n (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot avatar Nov 25 '25 16:11 Copilot

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

[!WARNING]

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • rcxvsblobprodcus328.vsblob.vsassets.io
    • Triggering command: /home/REDACTED/work/msbuild/msbuild/artifacts/bin/bootstrap/core/dotnet dotnet test src/Build.UnitTests/Microsoft.Build.Engine.UnitTests.csproj --filter FullyQualifiedName~TaskHost --no-build -v n (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot avatar Nov 25 '25 17:11 Copilot

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

[!WARNING]

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • rcxvsblobprodcus328.vsblob.vsassets.io
    • Triggering command: /home/REDACTED/work/msbuild/msbuild/artifacts/bin/bootstrap/core/dotnet dotnet test src/Build.UnitTests/Microsoft.Build.Engine.UnitTests.csproj --filter FullyQualifiedName~TaskHost --no-build -v n (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot avatar Nov 27 '25 12:11 Copilot

https://devdiv.visualstudio.com/DevDiv/_git/VS/pullrequest/695704 passed

JanProvaznik avatar Dec 16 '25 15:12 JanProvaznik