node-jq

node-jq copied to clipboard

Description

Invalid JQ Expression causes NestJS to completely crash. Attemp to catch exception is ignored by node.

Test Source

@Controller('/hello')
export class AppController {
  @Get('testjq')
  async testJq(): Promise<void> {
    await jq
      .run(
        'invalid expression',
        { test: 'object' },
        { input: 'json', output: 'compact' },
      )
      .then((res) => console.log(res))
      .catch((_err) => console.error('Trying to catch error'));
  }
}

Call URL Using HTTPie

$ http http://localhost:3000/hello/testjq

http: error: ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) while doing a GET request to URL: http://localhost:3000/hello/testjq

Error Message & Stack Trace

[Nest] 7974  - 06/29/2022, 2:52:49 AM     LOG [NestApplication] Nest application successfully started +2ms
Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:795:11)
    at Socket._write (node:net:807:8)
    at writeOrBuffer (node:internal/streams/writable:389:12)
    at _write (node:internal/streams/writable:330:10)
    at Socket.Writable.write (node:internal/streams/writable:334:10)
    at /home/user/projects/nestjsproject/node_modules/node-jq/lib/exec.js:25:27
    at new Promise (<anonymous>)
    at Object.exec [as default] (/home/user/projects/nestjsproject/node_modules/node-jq/lib/exec.js:15:12)

Environment

AlmaLinux 8 running inside WSL2 on Windows 11 (build 22000.778). Server using NestJS running on top of ExpressJS.

nodejs version:

Node.js v16.14.0
linux 5.10.102.1-microsoft-standard-WSL2

npm version:

8.3.1

node-jq version:

"version": "2.3.3",

NestJS info:

 _   _             _      ___  _____  _____  _     _____
| \ | |           | |    |_  |/  ___|/  __ \| |   |_   _|
|  \| |  ___  ___ | |_     | |\ `--. | /  \/| |     | |
| . ` | / _ \/ __|| __|    | | `--. \| |    | |     | |
| |\  ||  __/\__ \| |_ /\__/ //\__/ /| \__/\| |_____| |_
\_| \_/ \___||___/ \__|\____/ \____/  \____/\_____/\___/


[System Information]
OS Version     : Linux 5.10
NodeJS Version : v16.14.0
NPM Version    : 8.3.1

[Nest CLI]
Nest CLI Version : 8.2.8

[Nest Platform Information]
platform-express version : 8.4.7
schematics version       : 8.0.11
passport version         : 8.2.2
testing version          : 8.4.7
common version           : 8.4.7
config version           : 2.1.0
axios version            : 0.0.8
core version             : 8.4.7
jwt version              : 8.0.1
cli version              : 8.2.8

Hey @arinanto

The error that you show is EPIPE. It comes from node (https://nodejs.org/dist/latest-v12.x/docs/api/errors.html) and is saying that your code tries to write into a closed stream.

EPIPE (Broken pipe): A write on a pipe, socket, or FIFO for which there is no process to read the data. Commonly encountered at the net and http layers, indicative that the remote side of the stream being written to has been closed.

If you look closely to the error stack, you will see that node-jq runs at the beginning of the stack and it doesn't run any node:internal, neither relies on any streams.

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:795:11)
    at Socket._write (node:net:807:8)
    at writeOrBuffer (node:internal/streams/writable:389:12)
    at _write (node:internal/streams/writable:330:10)
    at Socket.Writable.write (node:internal/streams/writable:334:10)
--->    at /home/user/projects/nestjsproject/node_modules/node-jq/lib/exec.js:25:27 
    at new Promise (<anonymous>)
    at Object.exec [as default] (/home/user/projects/nestjsproject/node_modules/node-jq/lib/exec.js:15:12)

I managed to make a small repro, without nest with the minimum code:

const jq = require("node-jq");

const main = () => {
  jq.run(
    "invalid expression",
    { test: "object" },
    { input: "json", output: "compact" }
  )
    .then((res) => console.log(res))
    .catch((_err) => console.error("Trying to catch error"));
};

main();
/* node index.js */
/* Trying to catch error */

@davesnx thank you for the response, unfortunately a very simple ExpressJS app also crashed by your example code.

HTTPie:

$ http http://localhost:3000/testjq

http: error: ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) while doing a GET request to URL: http://localhost:3000/testjq

Output:

$ node server.js
Example app listening on port 3000
node:events:498
      throw er; // Unhandled 'error' event
      ^

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:795:11)
    at Socket._write (node:net:807:8)
    at writeOrBuffer (node:internal/streams/writable:389:12)
    at _write (node:internal/streams/writable:330:10)
    at Socket.Writable.write (node:internal/streams/writable:334:10)
    at /home/user/random/expressjq/node_modules/node-jq/lib/exec.js:25:27
    at new Promise (<anonymous>)
    at Object.exec [as default] (/home/user/random/expressjq/node_modules/node-jq/lib/exec.js:15:12)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
}

Entire server code:

const express = require('express')
const jq = require("node-jq");

const app = express()
const port = 3000

app.get('/testjq', async (req, res) => {
  await jq.run(
    "invalid expression",
    { test: "object" },
    { input: "json", output: "compact" }
  )
    .then((res) => console.log(res))
    .catch((_err) => console.error("Trying to catch error"));

  res.send('Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

package.json:

{
  "dependencies": {
    "express": "^4.18.1",
    "node-jq": "^2.3.3"
  }
}

I think the problem lies in the interaction between node-jq and ExpressJS. I've build a very simple ExpressJS app on my previous comment and it can reproduce the behaviour consistently.

Hey @arinanto

Can you try without any HTTP framework? The snippet above should work fine and discard any "fear" of being in the intersection between 2 technologies that have nothing to do with each other.

First, try that jq works in your system running: jq -r '.version' package.json

Later, try to run the simplest script. If that doesn't work, this issue should matter.

After trying to replicate the issue, I found the cause of your problems. Published a template to open bugs: https://replit.com/@DavidSancho/try-node-jq?v=1. You can try to replicate your issue there if you want.

If the "filter" contains a space, it will break node. So, if the expression is invalid, it will crash due a unhandledRejection in node. Making any script to explore if you don't handle that.

Thanks for clearing it up, I've just tried again and indeed it's because of the space.

I'm seeing the same crash with any invalid filter. E.g.: https://replit.com/@samarths-msft/try-node-jq#index.js. In our scenario, filter is supplied by ExpressJS app user. It can be invalid due to user error. If filter is invalid, we want to catch the error and return "invalid filter error" to the user. However, due to invalid filter, the ExpressJS app itself is crashing.

Meanwhile, I am looking into this. You can wrap the entire expression of jq with a try/catch, or add an unhandledRejection on the process.

This issue has been fixed since 2.3.5