Request reads from body stream twice, causing uncaught promise rejection due to unavailable writer

[huw]

Environment

Miniflare 2.0.0 running in VS Codespaces (Debian, Node 16)

Description/Reproduction

I ran into a pretty specific issue when trying to debug “Uncaught exceptions” with VS Code. This occurs on every fetch with a body parameter.

Exception has occurred: TypeError: Invalid state: Writer has been released
  at new NodeError (node:internal/errors:371:5)
    at writableStreamDefaultWriterRelease (node:internal/webstreams/writablestream:970:5)
    at finalize (node:internal/webstreams/readablestream:1218:5)
    at node:internal/webstreams/readablestream:1251:15
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

…which catches on this line in Undici.

Diagnosis

I traced it back through to Miniflare, which implicitly creates 2 Request (a.k.a. BaseRequest) objects. The first is at line 706 when we remove unwanted headers, and the second is at line 729, when we call through to Undici’s fetch. The problem is that Undici’s fetch then creates its own Request from the arguments we supply, on line 104.

I am sure that the problem is the two calls because editing 706-729 like this resolves the issue:

  // const req = new import_undici.Request(input, init);
  // req.headers[fetchSymbols.kGuard] = "none";
  // req.headers.delete("host");
  // req.headers.delete("cf-connecting-ip");
  // req.headers.set(_kLoopHeader, String(ctx?.requestDepth ?? 1));
  const removeHeaders = [];
  // for (const header of kDefaultHeadersToRemove) {
  //   if (!req.headers.has(header))
  //     removeHeaders.push(header);
  // }
  const dispatcher = new MiniflareDispatcher((0, import_undici.getGlobalDispatcher)(), removeHeaders);
  const baseRes = await baseFetch.call(dispatcher, input, init);

Impact

This makes debugging a little annoying, because I like to have it catch on uncaught exceptions, but as it stands I have to click through that exception on every request with a body (or mitigate as below).

Mitigation

You can click on the little pencil icon when hovering over Uncaught Exceptions and fill in error.name !== "TypeError" && error.message !== "Invalid state: Writer has been released", which filters out that specific error from catching.

Solution Suggestion

I assume we could fix this by not passing Undici’s own Request object to itself, even though Undici does accept that as a valid input param. We could probably do that by modifying the init object in-place to remove the headers, but at this point I don’t know enough about Miniflare’s internals to figure out if that would have any bad second-order effects.

Alternatively, this could be best passed on to Undici. But I also don’t really know enough about Node & Undici to phrase this properly.

I’d be happy to do the bulk of the work for either, with a little assistance! 😌

[mrbbot]

Hey! 👋 Thanks for raising this. It looks like this is an issue in undici. I see the same error with this example:

const { Request, fetch } = require("undici");

const req = new Request(
  "https://webhook.site/...", // replace with any POST-accepting endpoint
  {
    method: "POST",
    body: "body",
  }
);

fetch(req).then((res) => {
  console.log(res.headers);
});

I'd suggest opening a new bug report there with this reproduction. They're very friendly, and you definitely know enough about Node & Undici given the detail you've provided in this issue. 🙂

[mrbbot]

Hey! 👋 Happy new year! 😅 Were you able to open a bug report in undici for this?

[huw]

Sorry mate, I didn’t. I ended up going with the workaround I mentioned above. Happy to raise one or +1 it if you want.

[mmaha-CC]

I'm having the same issue. I used your workaround but it's pretty annoying to do that iykwim. Did you manage to raise the issue with Undici?

[mrbbot]

Hey! 👋 Thanks for raising this, and apologies for the very delayed response. This should be fixed in Miniflare 3, which uses the same workerd runtime as production, so shouldn't have these behaviour mismatches. I'm going to close this as it's unlikely we'll have time to fix this in Miniflare 2. Our development effort is primarily focused on version 3.