n8n
n8n copied to clipboard
Support ndjson for application/json requests
Backstory -
I have a workflows that starts with a webhook.
This webhook is called with a request with Content-Type: application/json
, and ndjson body, like this:
{ "id": 1, "message": "hello" }
{ "id": 2, "message": "I'm a JSON inside a NDJSON" }
I am completely unable to make the webhook work, because the parsing of the json is happening in Server.ts
, so fiddling with the workflow settings (e.g, setting rawBody
to true) - has no effect.
I get an error in the log, and the workflow is not started.
What I propose in this PR -
If a request is made with Content-Type: application/json
, and parsing the body as json fails -
try to parse it as ndjson.
- If that works - great! return an array of all the jsons in the body (1 line = 1 item)
- if not - log an error, and pass the original answer.
Note:
In a perfect world, the API I'm consuming would send the request to the webhook with application/x-ndjson
.
But I have no control over that - it's a 3rd party. And that would be the experience of most users of n8n.
I also tried to see what would've happened if the Content-Type was right - I got an empty body.
This PR is related to this feature-request from body-parser: https://github.com/expressjs/body-parser/issues/478 (If that feature-request is handled, then we could use it here instead of this PR. But I don't know if they'd answer)
General questions to this approach:
- what if you cant determine the type of the request body based on the Content-Type header (for example, incorrect header? or if the type is generic)
General questions to this approach:
- what if you cant determine the type of the request body based on the Content-Type header (for example, incorrect header? or if the type is generic)
If we can't determine the type of the request body based on the Content-Type header, then we continue as usual, no change there. I believe the default body type is plain text. If that causes an error - very well. An incorrect Content-Type is a serious error by the client.
I've accepted CR comments, and rebased this branch.
I've since stopped using n8n, so I will not make further changes to this PR. I hope this PR can be of help to others.
Sorry that this never got reviewed or merged. Unfortunately this can't be rebased anymore, since Server.ts
has changed quite a lot.
If anyone needs ndjson support on webhooks, you can enable the "Binary Data" option on the webhook, and then parse this data in a "Code" node.
If you think this should actually be a feature in n8n, then please create a Feature request, so that this can first be discussed with the team before anything is implemented 🙏🏽