nx icon indicating copy to clipboard operation
nx copied to clipboard

feat(core): resilient daemon restart with exponential backoff reconnection

Open FrozenPandaz opened this issue 1 month ago • 3 comments

Current Behavior

The daemon currently depends on client requests failing with a LOCK_FILES_CHANGED error to trigger a restart. This creates several issues:

  • The daemon may stay running with stale dependencies if no requests come in
  • The client must wait for a request to fail to trigger reconnection
  • Special-case error handling is scattered across the client code

Expected Behavior

The daemon should proactively restart itself when lock files change, and the client should gracefully reconnect with exponential backoff for any server shutdown scenario.

Changes

Client-Side: Exponential Backoff Reconnection

  • Add handleConnectionError() method that retries with exponential backoff (10ms → 5000ms, 30 attempts max)
  • Preserve pending messages during reconnection and resend them once the new daemon is available
  • Remove special-case handling for LOCK_FILES_CHANGED and NX_VERSION_CHANGED errors
  • Remove retryMessageAfterNewDaemonStarts() method as it's no longer needed

Server-Side: Self-Restart on Lock File Changes

  • Add startNewDaemonInBackground() to spawn a replacement daemon before shutdown
  • Add handleServerProcessTerminationWithRestart() for restartable shutdown scenarios
  • Detect lock file changes and proactively start a new daemon before responding with an error
  • Keep version change handling simple (just exit, no restart)

Benefits

  • More Resilient: Client recovers from any server shutdown, not just specific errors
  • Cleaner Architecture: Server manages its own lifecycle, client doesn't need special cases
  • No Request Dependency: Daemon doesn't wait for requests to detect changes
  • Reduced Error Spam: Exponential backoff prevents connection error floods
  • Future-Proof: Foundation for other restart scenarios (plugins, config changes)

Related Issue(s)

Part of daemon rework to make restarts more reliable and resilient.

FrozenPandaz avatar Nov 10 '25 22:11 FrozenPandaz

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
nx-dev Ready Ready Preview Dec 18, 2025 2:23am

vercel[bot] avatar Nov 10 '25 22:11 vercel[bot]

Deploy Preview for nx-docs ready!

Name Link
Latest commit d85625442e8afb92d0c0e3dcad63da6ca5a2e907
Latest deploy log https://app.netlify.com/projects/nx-docs/deploys/6943643e28e12300083db56a
Deploy Preview https://deploy-preview-33432--nx-docs.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify[bot] avatar Nov 10 '25 22:11 netlify[bot]

View your CI Pipeline Execution ↗ for commit d85625442e8afb92d0c0e3dcad63da6ca5a2e907

Command Status Duration Result
nx affected --targets=lint,test,test-kt,build,e... ✅ Succeeded 4m 1s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 1m 47s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 7s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 3s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2025-12-18 03:58:44 UTC

nx-cloud[bot] avatar Nov 11 '25 21:11 nx-cloud[bot]