node icon indicating copy to clipboard operation
node copied to clipboard

Enable passthrough IPC in watch mode

Open znewsham opened this issue 1 year ago • 33 comments

Fixes https://github.com/nodejs/node/issues/50880

When spawning a node process in watch mode, from another node process, there is currently no way to get IPC between the parent and the child. This PR enables this bi-directional flow:

  • parent -> watcher -> child
  • child -> watcher -> parent

This setup is useful in situations where you have a process responsible for building and running a client/server application which needs to be made aware (via IPC) that it's client bundle has changed and needs to be re-served, without restarting the server.

znewsham avatar Nov 24 '23 04:11 znewsham

I think this should be directed towards the main branch

debadree25 avatar Nov 24 '23 05:11 debadree25

I think this should be directed towards the main branch

I confess, I'm not totally clear on the branching structure. I'd like this to be available in node 18, are you suggesting I point at v18.x or somewhere else?

znewsham avatar Nov 24 '23 05:11 znewsham

So the general process is we land changes onto the main branch of this repo and then changes are backported to the release lines you can read about it here https://github.com/nodejs/node/blob/main/doc/contributing/backporting-to-release-lines.md so for your case your branch should towards the main branch and after this change is approved and landed it shall be backported by the releasers to the 18.x branch and be available in 18.x

debadree25 avatar Nov 24 '23 05:11 debadree25

please also resolve conflicts

MoLow avatar Nov 24 '23 11:11 MoLow

This will almost certainly require a backport PR

noting the original commit for my reference later: ad2131716bdd45d4279a9780f50e11e1aa4302ef

znewsham avatar Nov 24 '23 14:11 znewsham

Both of those are fixed - for some reason I can't get eslint to play nice with vscode - but the linting is now resolved.

znewsham avatar Nov 25 '23 18:11 znewsham

Just want to check on the next steps here - the contribution guide suggests I should kick off the CI build and add an "Author Ready" label - I don't believe I have permissions to do either

znewsham avatar Dec 04 '23 16:12 znewsham

CI: https://ci.nodejs.org/job/node-test-pull-request/56076/

nodejs-github-bot avatar Dec 04 '23 16:12 nodejs-github-bot

What's the best way to debug failures on specific architectures? E.g., node-test-commit-arm-fanned node-test-commit-linuxone-rhel8-s390x

I could probably get myself access to an arm machine if necessary

znewsham avatar Dec 04 '23 17:12 znewsham

CI: https://ci.nodejs.org/job/node-test-pull-request/56582/

nodejs-github-bot avatar Dec 29 '23 17:12 nodejs-github-bot

the failures look unrelated, re-running

MoLow avatar Jan 02 '24 20:01 MoLow

CI: https://ci.nodejs.org/job/node-test-pull-request/56629/

nodejs-github-bot avatar Jan 02 '24 20:01 nodejs-github-bot

@znewsham please take a look at the windows failures: https://ci.nodejs.org/job/node-test-binary-windows-js-suites/25276/


22:34:45         # Subtest: should pass IPC messages from a spawning parent to the child and back
22:34:45         not ok 20 - should pass IPC messages from a spawning parent to the child and back
22:34:45           ---
22:34:45           duration_ms: 655.1977
22:34:45           location: 'file:///C:/workspace/node-test-binary-windows-js-suites/node/test/sequential/test-watch-mode.mjs:378:3'
22:34:45           failureType: 'testCodeFailure'
22:34:45           error: |-
22:34:45             Expected values to be strictly deep-equal:
22:34:45             + actual - expected
22:34:45             
22:34:45               [
22:34:45                 'running',
22:34:45                 'Received: first message',
22:34:45             -   "Restarting 'C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.960\\25.js'",
22:34:45             -   'running',
22:34:45                 'Received: second message',
22:34:45                 "Completed running 'C:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.960\\\\25.js'"
22:34:45               ]
22:34:45           code: 'ERR_ASSERTION'
22:34:45           name: 'AssertionError'
22:34:45           expected:
22:34:45             0: 'running'
22:34:45             1: 'Received: first message'
22:34:45             2: "Restarting 'C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.960\\25.js'"
22:34:45             3: 'running'
22:34:45             4: 'Received: second message'
22:34:45             5: "Completed running 'C:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.960\\\\25.js'"
22:34:45           actual:
22:34:45             0: 'running'
22:34:45             1: 'Received: first message'
22:34:45             2: 'Received: second message'
22:34:45             3: "Completed running 'C:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.960\\\\25.js'"
22:34:45           operator: 'deepStrictEqual'
22:34:45           stack: |-
22:34:45             TestContext.<anonymous> (file:///C:/workspace/node-test-binary-windows-js-suites/node/test/sequential/test-watch-mode.mjs:447:12)
22:34:45             process.processTicksAndRejections (node:internal/process/task_queues:95:5)
22:34:45             async Test.run (node:internal/test_runner/test:632:9)
22:34:45             async Promise.all (index 19)
22:34:45             async Suite.run (node:internal/test_runner/test:949:7)
22:34:45             async startSubtest (node:internal/test_runner/harness:218:3)
22:34:45           ...

MoLow avatar Jan 02 '24 20:01 MoLow

I don't have a windows machine either :D but I can probably get access to one

znewsham avatar Jan 02 '24 20:01 znewsham

@MoLow Would you mind kicking off the CI job again, I believe this was a simple race condition in the test

znewsham avatar Jan 18 '24 16:01 znewsham

CI: https://ci.nodejs.org/job/node-test-pull-request/56838/

nodejs-github-bot avatar Jan 18 '24 19:01 nodejs-github-bot

@MoLow apart from a silly linting error which I just fixed, I think this passed CI? The windows tests seemed to pass this time

znewsham avatar Jan 19 '24 18:01 znewsham

the change to the test seem to increase flakyness

MoLow avatar Jan 20 '24 20:01 MoLow

@MoLow can you expand on that a little? I'm seeing "All checks have passed" (in github) - seems to be a success, I looked into the build, it does say it's failed - but the only relevant one I can see appears to be related to linting (which I've fixed).

znewsham avatar Jan 21 '24 00:01 znewsham

Sure, see this Jenkins run: https://ci.nodejs.org/job/node-test-binary-windows-js-suites/25477/

MoLow avatar Jan 22 '24 10:01 MoLow

Thanks for the link - I'm not familiar with Jenkins at all, can you briefly explain:

  1. how you got there from the link the bot posted ( https://ci.nodejs.org/job/node-test-pull-request/56838/)
  2. how I can see logs of the failing tests? All I can see is "flaky (1)" and the stacktrace is equally useless

I'm working on being able to reproduce these locally - can you confirm (as it seems I can't!) that it's just the windows build having problems?

znewsham avatar Jan 22 '24 15:01 znewsham

@MoLow re-ci please! I don't believe my change made it more flaky, but there was still a (trivial) bug in the test.

        +   "Restarting 'C:\\\\Users\\\\REDACTED\\\\code\\\\node\\\\test\\\\.tmp.3930\\\\24.js'",
        -   "Restarting 'C:\\Users\\REDACTED\\code\\node\\test\\.tmp.3930\\24.js'",

znewsham avatar Jan 22 '24 20:01 znewsham

Thanks for the link - I'm not familiar with Jenkins at all, can you briefly explain:

  1. how you got there from the link the bot posted ( ci.nodejs.org/job/node-test-pull-request/56838)
  2. how I can see logs of the failing tests? All I can see is "flaky (1)" and the stacktrace is equally useless

I'm working on being able to reproduce these locally - can you confirm (as it seems I can't!) that it's just the windows build having problems?

inside https://ci.nodejs.org/job/node-test-pull-request/56838/ click on any of the items that have a yellow circle near them (that means they have flaky tests), so I clicked on node-test-binary-windows-js-suites and saw the results. logs can be viewd when clickin on Console Output on the left

MoLow avatar Jan 23 '24 20:01 MoLow

Thanks @MoLow just waiting for CI now

znewsham avatar Jan 25 '24 23:01 znewsham

CI: https://ci.nodejs.org/job/node-test-pull-request/56926/

nodejs-github-bot avatar Jan 26 '24 07:01 nodejs-github-bot

This needs a rebase to solve the git conflicts.

aduh95 avatar May 05 '24 08:05 aduh95

@aduh95 I have completed the rebase. Tests are passing locally for me. A new CI run would be appreciated

znewsham avatar May 06 '24 18:05 znewsham

Failed to start CI
   ⚠  Something was pushed to the Pull Request branch since the last approving review.
   ✘  Refusing to run CI on potentially unsafe PR
https://github.com/nodejs/node/actions/runs/8974210560

github-actions[bot] avatar May 06 '24 18:05 github-actions[bot]

Not sure if the error above is because of the force push necessary for a rebase?

znewsham avatar May 06 '24 18:05 znewsham

@aduh95 resolved the linter errors - could you kick off another CI run please

znewsham avatar May 07 '24 12:05 znewsham