Avoid address-already-in-use errors when using watch & inspect

[stephenh]

Feature request

When using tsx watch --inspect=..., when files change, the node debug server is briefly stopped & restarted.

This makes sense, but if I've attached a debugger in Webstorm, I'd ideally like the debugger to stay connected across restarts / as I'm editing and changing files.

Currently if I tell Webstorm debugger to "Reconnect automatically" on disconnect, it seems to grab the port 9229 before the tsx's restarted node takes it, so I get an error:

  Starting inspector on 0.0.0.0:9229 failed: address already in use

Why?

I'd like to have a really pleasant "the debugger stays connected and it just works across restarts" workflow.

Alternatives

I'd used ts-node-dev in the past, and I swear they had this working, i.e. somehow webstorm could auto reconnect across restarts, without getting "port in use" conflicts.

Maybe I'm just wrong/making this up though, b/c I'm not sure how they would prevent this, without running their own proxy on the port that forwards to the currently-booted node process...

Otherwise, to work around this, I'm just disabling the webstorm auto-reconnect, and I have to remember to keep clicking "connect debugger" after time I change a file.

Additional context

No response

[stephenh]

Fwiw I've confirmed that, per my recollection, ts-node-dev (via node-dev itself afaiu) is able to restart and have the debugger seamlessly reconnected.

It looks like this was fixed in node-dev in ~2015 by "graceful restarts" in their 2.2.0 release:

https://github.com/fgnass/node-dev/issues/80#issuecomment-133371928

Looking at the diffs between 2.1.0 and 2.2.0 nothing in particular stands out as "graceful restarts":

https://github.com/fgnass/node-dev/compare/v2.1.0...v2.2.0

And AFAICT they are not doing anything super fancy on restart:

https://github.com/fgnass/node-dev/blob/5d07f002cb9288450ebea1dcc9f60205c958897c/lib/index.js#L145

I.e. they just sent child.kill(SIGTERM)...

[stephenh]

Maybe it's because node-dev uses child_process.fork?

https://github.com/fgnass/node-dev/blob/5d07f002cb9288450ebea1dcc9f60205c958897c/lib/index.js#L117

[privatenumber]

I'm not familiar with using the debugger in Node.js so it will likely be a while before I can investigate this.

Feel free to open a PR if you want.

[stephenh]

Just a note of progress, I poked at this a little bit over the weekend and noticed that in a small/tiny app, with tsx watch+inspect and a single index.ts file, I actually can't reproduce the "address already in use" error, everything seems to work great.

It's only when I use tsx watch+inspect in our large production codebase that something in the timing gets off, and the port reuse error happens (but that using node-dev/ts-node-dev on our large codebase somehow works fine).

Disclaimer that even in our codebase, tsx will reload and reuse the port correctly maybe ~30% of the time (so the "address already in use" error happens maybe ~70% of the time), so the non-determinism makes it hard to be 100% confident in my assertions.

I'll keep poking at this as I find time.

[dunckr]

This happened to me. You need to handle SIGTERM and close your server so the port is released e.g.:

process.on('SIGTERM', async () => {
    await server.close();
});

await server.listen()

[stephenh]

Hey @dunckr ! Thanks for the hint; just curious, was your await server.close() fixing "port in use errors" on like your express/fastify/etc port 3000 or 4000 or what not, or was it also fixing "port in use errors" for the debug port 9229?

My initial guess is that doing await server.close() would fix the former, but I'm having issues with the latter.

[RedCrafter07]

For me, it isn't resolved. The app even crashes in normal mode. (tsx index.ts) I'm using an Express Server, my OS is Windows. (if that matters) Any suggestions? On my custom script consisting of a concurrently script running typescript in watch mode and then restart the server using nodemon because of JS changes works without any issues. I'd love to use tsx in my own project, however this is what prevents me of doing so sadly. :/

[privatenumber]

@stephenh @dunckr Note that tsx just passes on the CLI flags to Node.js, so it's actually Node.js that's handling it.

You might be able to investigate what sockets need to be closed on SIGTERM using https://github.com/mafintosh/why-is-node-running.

Something worth considering is waiting till the previous child process emits exit before starting the next one here: https://github.com/esbuild-kit/tsx/blob/develop/src/watch/index.ts#L66

@RedCrafter07 Please open a separate issue. Sounds like it's not directly relevant to this issue, and I'm interested in debugging a reproduction.

[stephenh]

I've been working with the latest releases, and it seems like https://github.com/esbuild-kit/tsx/pull/116 fixed this!

I'm able to see tsx watch pick up reloads while connected with a debugger and everything works great, the IDE auto-reconnects to the restarted Node process/etc.

So going to close this out. Thanks @privatenumber !

[u3u]

I also encountered this problem. I just need to open the browser's node debugger, then edit any file and save it (it seems to be no problem when I don't open the debugger)

Run tsx watch --inspect src/server.ts on "tsx": "^3.12.6"

import dotenv from 'dotenv'
import { fastify } from 'fastify'

dotenv.config()

const server = fastify({ logger: false })

server.register(import('./routes'))

const start = async () => {
  try {
    const port = +(process.env.PORT || 3000)
    await server.listen({ port })
    console.log(`started server on http://localhost:${port}`)
  } catch (err) {
    console.error(err)
    process.exit(1)
  }
}

Even listening to SIGTERM is useless.

const stop = async () => {
  console.log('stopping server...')
  await server.close()
  process.exit(0)
}

process.on('SIGINT', stop)
process.on('SIGTERM', stop)

[privatenumber]

Note that process.on() doesn't accept async handlers, so using one prevents it from actually running at the correct timing.

[u3u]

I tried it out and even without using async, there are still issues. I have to manually close the Node debugger window in the browser to release the port.

[LeSphax]

I'm running into the same problem.

After some debugging I've been able to reproduce with the following script:

console.log(process.pid + ': Starting');
const shutdown = (code) => {
  console.log(process.pid + `: ${code} received.`);
  process.exit(0);
};
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
process.on('SIGTERM', () => console.log('SIGTERM received'));

function fibonacci(n) {
  if (n <= 1) {
    return n;
  }
  return fibonacci(n - 1) + fibonacci(n - 2);
}

(async function main() {
  while (true as any) {
    const n = 45; // Adjust this value to increase/decrease the workload
    console.log(process.pid + `:Calculating Fibonacci(${n})...`);
    const result = fibonacci(n);
    console.log(process.pid + `: Fibonacci(${n}) = ${result}`);
    await new Promise(resolve => setTimeout(resolve, 100));
  }
})();

export default fibonacci;

And running it like that: yarn tsx watch --inspect=0.0.0.0:9229 --clear-screen=false script.ts

You end up with something like this:

yarn tsx watch --inspect=0.0.0.0:9229 --clear-screen=false config/shutdown.ts
Debugger listening on ws://0.0.0.0:9229/7a65e8ad-8624-4b69-840a-12d4a67baa32
For help, see: https://nodejs.org/en/docs/inspector
94566: Starting
94566:Calculating Fibonacci(45)...
4:40:56 PM [tsx] rerunning
Starting inspector on 0.0.0.0:9229 failed: address already in use
94578: Starting
94578:Calculating Fibonacci(45)...
94566: Fibonacci(45) = 1134903170
94566: SIGTERM received.

I'm guessing this is something about node not going back to the event loop while it's running heavy calculations. Which causes it to not register the SIGTERM message in time. Meaning the next process ends up starting before the last one stopped.

In my actual codebase I think this happens when it's initially importing all the modules.

Changing runProcess.kill(); to runProcess.kill("SIGKILL"); seems to fix it for that example. So maybe that could be an optional parameter of tsx watch? I don't really need my process to exit gracefully when doing local development.

[foofel]

still have the same problem with 3.12.7

[privatenumber]

I think the problem you guys are referring to will be fixed by https://github.com/esbuild-kit/tsx/pull/244

[GMaiolo]

Can we reopen this until #244 is merged and published? Had to switch to ts-node until that happens

[stephenh]

Fwiw maybe a good way to encourage @privatenumber to have time to look at #244 is sponsoring him; I started a $10/month sponsorship that I'll let run for at least 6 months, potentially longer/will see, perhaps others blocking on this issue could do the same.

I use tsx everywhere these days, so thanks/appreciate it @privatenumber !

[privatenumber]

Really appreciate your support @stephenh!

I work on tsx on my spare time so it's not always easy to find the time. Sponsorship aligns the incentives so it makes sense to work on issues that are not directly relevant to me.

[GMaiolo]

I completely agree with the sponsorship incentive, and thank you @privatenumber for the beautiful project that's tsx!

I'm not asking to rush for the fix, but just reopening this issue to align with the current state of the project. This issue was hard to find as it's marked as closed when it's not yet fixed, as far as I understand.