nx:run-commands hangs when running multiple commands sequentially

[ejasarevic]

Current Behavior

Having a simple target inside project.json -> targets

"test": {
      "executor": "nx:run-commands",
      "options": {
        "commands": [
          "echo 'test 1'",
          "echo 'test 2'"
        ],
        "parallel": false
      }
    },

It only prints 'test 1' and hangs indefinitely. It is not possible to stop the process. Tested it with NX 17.3.2 and it works as expected with that version.

Expected Behavior

Both commands are executed sequentially (non parallel). Output is: test 1 test 2

GitHub Repo

No response

Steps to Reproduce

1. Create new NX App with NX 18.x
2. Add a new target to project.json with executor "executor": "nx:run-commands",
3. Set "parallel: false"
4. Add any 2 commands inside "options" -> "commands" array

Nx Report

Node   : 18.18.0                                         
OS     : win32-x64                                       
npm    : 9.8.1

   nx                 : 18.0.3
   @nx/js             : 18.0.3
   @nx/jest           : 18.0.3
   @nx/linter         : 18.0.3
   @nx/eslint         : 18.0.3
   @nx/workspace      : 18.0.3
   @nx/devkit         : 18.0.3
   @nx/eslint-plugin  : 18.0.3
   @nx/next           : 18.0.3
   @nx/playwright     : 18.0.3
   @nx/react          : 18.0.3
   @nrwl/tao          : 18.0.3
   @nx/web            : 18.0.3
   @nx/webpack        : 18.0.3
   typescript         : 5.3.3

Failure Logs

No response

Package Manager Version

9.8.1

Operating System

• [X] macOS
• [X] Linux
• [X] Windows
• [ ] Other (Please specify)

Additional Information

No response

[danhobbs75]

I am experiencing this with Nx 18.0.4. I tried a workaround where I split a prebuild step into two where each has a single command instead of an array of commands, and now my build dependsOn ["prebuild1","prebuild2"]. The workaround is fine for me in a simple case of running two commands. HOWEVER, the build step then hangs. I can run each individually but obviously this is not ideal.

[kwextpb]

Not really a workaround and it scales horribly, cause if you had let's say 7 commands you would need to add 7 targets with a lot of syntax sugar. As a workaround I would suggest for now downgrade nx to 17.

[danhobbs75]

Not really a workaround and it scales horribly, cause if you had let's say 7 commands you would need to add 7 targets with a lot of syntax sugar. As a workaround I would suggest for now downgrade nx to 17.

You are quite right that it doesn't scale, and I only mention it to back up that I've seen the same behaviour, and in case anyone else only has two commands and is happy to wait for a fix by doing what I'm doing (in fact, only one command with a chained dependsOn seems to have the same problem as I see you've mentioned elsewhere - behaviour I also see).

[Bonscho]

Can confirm the issue. Are there news regarding a workaround or solution? Otherwise we would need to downgrade ;(

[rockse]

We are facing the same issue

[Cammisuli]

I'll look into this as soon as possible. In the meantime, can you confirm that things work as expected when setting the env var NX_NATIVE_COMMAND_RUNNER=false?

[danhobbs75]

I'll look into this as soon as possible. In the meantime, can you confirm that things work as expected when setting the env var NX_NATIVE_COMMAND_RUNNER=false?

Same result for me: hangs when executing my original prebuild task which has two commands.

[Ionaru]

@Cammisuli

Using NX_NATIVE_COMMAND_RUNNER=false does fix the issue for me with the simpler task syntax.

  "targets": {
    "some-task": {
      "command": "echo 'Command 1'"
    },
  ...

And it also seems to work when I use the nx:run-commands syntax.

  "targets": {
    "some-task": {
      "executor": "nx:run-commands",
      "options": {
        "commands": [
          "echo 'Command 1'",
          "echo 'Command 2'"
        ],
        "parallel": false
      }
    },
  ...

[Bonscho]

I can confirm that setting the NX_NATIVE_COMMAND_RUNNER env to false solves the problem for us with the following target setup.

"copy-assets": {
	"executor": "nx:run-commands",
	"options": {
		"commands": [
			{
				"command": "some command",
				"forwardAllArgs": false
			},
			{
				"command": "some other command",
				"forwardAllArgs": false
			}
		],
		"parallel": false
	}
}

and

"build": {
	"executor": "@nx/webpack:webpack",
	"outputs": ["{options.outputPath}"],
	"dependsOn": [
		{
			"target": "copy-assets"
		},
		…
	], 
	…
}

[danhobbs75]

I'll look into this as soon as possible. In the meantime, can you confirm that things work as expected when setting the env var NX_NATIVE_COMMAND_RUNNER=false?

Same result for me: hangs when executing my original prebuild task which has two commands.

Actually, I'm going to rephrase my answer having seen that @Bonscho had success. The task is running but I get no output except the result of the Boolean expression to run two commands in Powershell:

($env:NX_NATIVE_COMMAND_RUNNER = "false") -and (nx prebuild)

The above outputs true, but I can tell that the prebuild task HAS been executed but I can't see the output

Then I get this result when I try to run one of two commands on its own via a separate Nx run-commands target (nx prebuild1 - where I have the first of the two commands on its own):

 >  NX   Nx Cloud Problems

   Error: write EPIPE

Messages from the log:
[NX Daemon Server] - 2024-02-19T09:05:07.068Z - Done sending EndRun request
[NX Daemon Server] - 2024-02-19T10:00:47.982Z - Established a connection. Number of open connections: 1
[NX Daemon Server] - 2024-02-19T10:00:47.982Z - Closed a connection. Number of open connections: 0
[NX Daemon Server] - 2024-02-19T10:00:47.983Z - Established a connection. Number of open connections: 1
[NX Daemon Server] - 2024-02-19T10:00:47.988Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2024-02-19T10:00:47.988Z - [REQUEST]: Responding to the client. project-graph
[NX Daemon Server] - 2024-02-19T10:00:47.991Z - Time taken for 'total for creating and serializing project graph' 0.31610000133514404ms
[NX Daemon Server] - 2024-02-19T10:00:47.992Z - Done responding to the client project-graph
[NX Daemon Server] - 2024-02-19T10:00:47.992Z - Handled REQUEST_PROJECT_GRAPH. Handling time: 0. Response time: 4.
[NX Daemon Server] - 2024-02-19T10:00:48.092Z - [REQUEST]: Responding to the client. handleHashTasks
[NX Daemon Server] - 2024-02-19T10:00:48.093Z - Done responding to the client handleHashTasks
[NX Daemon Server] - 2024-02-19T10:00:48.093Z - Handled HASH_TASKS. Handling time: 31. Response time: 1.
[NX Daemon Server] - 2024-02-19T10:00:50.168Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2024-02-19T10:00:50.170Z - [REQUEST]: Responding to the client. recordOutputsHash
[NX Daemon Server] - 2024-02-19T10:00:50.170Z - [WATCHER]: Stopping the watcher for D:\GitHub\bb (sources)
[NX Daemon Server] - 2024-02-19T10:00:50.170Z - Done responding to the client recordOutputsHash
[NX Daemon Server] - 2024-02-19T10:00:50.170Z - Handled RECORD_OUTPUTS_HASH. Handling time: 0. Response time: 0.
[NX Daemon Server] - 2024-02-19T10:00:50.170Z - [WATCHER]: Stopping the watcher for D:\GitHub\bb (outputs)
[NX Daemon Server] - 2024-02-19T10:00:50.170Z - Server stopped because: "Lock file changed"



More information: D:\GitHub\bb\.nx\cache\d\daemon.log

So something is not happy. I get this error report and log snippet every time I run an Nx task subsequently. The prebuild1 task which has got one command does always run successfully, and I can see its output in the console window, but I also get that bonus error report.

UPDATE:

I changed the command I'm running to this and now I DO see the output:

$env:NX_NATIVE_COMMAND_RUNNER = "false" ; nx prebuild  

I'm not a Powershell expert, it's fair to say. :-)

Still getting the error report if I run the prebuild1 task with the single command.

[bubblegumsoldier]

@Cammisuli

Using NX_NATIVE_COMMAND_RUNNER=false does fix the issue for me with the simpler task syntax.

  "targets": {
    "some-task": {
      "command": "echo 'Command 1'"
    },
  ...

And it also seems to work when I use the nx:run-commands syntax.

  "targets": {
    "some-task": {
      "executor": "nx:run-commands",
      "options": {
        "commands": [
          "echo 'Command 1'",
          "echo 'Command 2'"
        ],
        "parallel": false
      }
    },
  ...

I can confirm this works for me.

However I was trying to be a smartypants and did this

        "start": {
            "executor": "nx:run-commands",
            "dependsOn": ["build"],
            "options": {
                "commands": ["my-command"],
                "env": {
                    "NX_NATIVE_COMMAND_RUNNER": "false"
                },
                "parallel": false
            }
        }

But that did not work

[Cammisuli]

Hmm.. I tried giving this a shot locally with doing echos and it looks like things are completing properly for me. Whats another command where you all noticed this was hanging with?

Here's a screenshot of my local run:

[Ionaru]

@Cammisuli You are running the target directly. The problem (at least, for me), exists when you have your temp target in a dependsOn of another target.

Add "dependsOn": ["temp"] in your build target and run parallel-test:build.

[danhobbs75]

Hmm.. I tried giving this a shot locally with doing echos and it looks like things are completing properly for me. Whats another command where you all noticed this was hanging with?

My prebuild target looks like this:

    "prebuild": {
      "executor": "nx:run-commands",
      "outputs": [],
      "options": {
        "commands": [
          "echo 'prebuild'",
          "npm --no-git-tag-version version patch",
          "echo 'version patch done'",
          "node extract-version.js",
          "echo 'extract-version.js done'"
        ],
        "parallel": false
      }

My build dependsOn it.

But also I separated them so this will run fine if targeted directly:

    "prebuild1": {
      "executor": "nx:run-commands",
      "outputs": [],
      "options": {
        "command": "npm --no-git-tag-version version patch"
      }
    },

...but the moment I make the build depend on that prebuild1 target, it also hangs.

I hope that helps you track it down. Sounds like an annoying "works for me" one which is difficult to replicate. :-(

[Cammisuli]

Ok I’ll check again with that.

[JakeGinnivan]

My colleague had issues with some WSL process running away.

I wonder if it's something to do with using Terminal (non-wsl) then behind the scenes NX is shelling commands which are running in WSL and something between the two environments is going bad.

When the issue happened for him, this process went runaway.

It kept growing in memory usage and was sustaining ~100MB/s disk usage until his machine grinds to a halt and needs a hard reset.

[Inkdpixels]

Just chiming in since we just experienced quite a similar issue but I have no additional information wether it is a separate one. We also have targets with sequential nx:run-commands which fail to execute / hang after 1-2 commands.

Funnily enough, once we add --output-style=stream, the commands succeed.

So TL;DR

npx nx run <project>:<target> --verbose                         # fails
npx nx run <project>:<target> --verbose --output-style=stream   # succeeds

Also interestingly, I could not observe the failures when invoking the target through nx run-many. Maybe this helps anyone?

Node   : 20.10.0
OS     : win32-x64
npm    : 10.2.3

nx                 : 18.0.5
@nx/js             : 18.0.5
@nx/jest           : 18.0.5
@nx/linter         : 18.0.5
@nx/eslint         : 18.0.5
@nx/workspace      : 18.0.5
@nx/devkit         : 18.0.5
@nx/eslint-plugin  : 18.0.5
@nx/express        : 18.0.5
@nx/node           : 18.0.5
@nx/plugin         : 18.0.5
@nx/react          : 18.0.5
@nrwl/tao          : 18.0.5
@nx/web            : 18.0.5
@nx/webpack        : 18.0.5
typescript         : 5.3.3

[atamike]

I ran today into the same problem. We have updated from NX17 to NX18 and the problem occurs. My collegues which are using mostly Mac are fine, the problem is only for me and those who are using Windows.

[BenPag]

One other fix, that works for me, is the following and after that everything works fine under windows.

Adding NX_NATIVE_COMMAND_RUNNER=false to the .env file at workspace root. The .env file was introduced with NX18.

[danhobbs75]

One other fix, that works for me, is the following and after that everything works fine under windows.

Adding NX_NATIVE_COMMAND_RUNNER=false to the .env file at workspace root. The .env file was introduced with NX18.

This is a great workaround and also works for me (running on Windows 11). Thanks for sharing this one.

[MoeedthePessimist]

I am having the same issue with building my react frontend. The build commands hangs at one point after builiding all the libraries. I have also tried using the command after adding NX_NATIVE_COMMAND_RUNNER=false to the root .env.

I have generated a ticket but haven't gotten any leads on it. #22094

[bsplosion]

One other fix, that works for me, is the following and after that everything works fine under windows.

Adding NX_NATIVE_COMMAND_RUNNER=false to the .env file at workspace root. The .env file was introduced with NX18.

Thanks so much for mentioning this. I was struggling to get NX to recognize the user environment copy of that variable but this solved it cleanly.

I assume once https://github.com/nrwl/nx/pull/22294 is merged and released we can remove that env variable again? Surprised this one wasn't caught during regression tests, seems like a very common use case.

[github-actions[bot]]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.