🐛 Bug Report — Runtime APIs: `Can't read from request stream after response has been sent.` in Durable Object when `request.body` not consumed

[mrbbot]

Hey! 👋

With the following configuration...

using Workerd = import "/workerd/workerd.capnp";

const config :Workerd.Config = (
  services = [
    ( name = "main", worker = .worker ),
  ],
  sockets = [
    ( name = "http", address = "*:8080", http = (), service = "main" ),
  ]
);

const worker :Workerd.Worker = (
  compatibilityDate = "2023-07-24",
  modules = [ ( name = "index.mjs", esModule = embed "index.mjs" ) ],
  durableObjectNamespaces = [
    ( className = "TestObject", uniqueKey = "TestObject" )
  ],
  durableObjectStorage = ( inMemory = void ),
  bindings = [
    ( name = "OBJECT", durableObjectNamespace = "TestObject" ),
  ],
);

// index.mjs
export class TestObject {
  async fetch(request) {
    const { pathname } = new URL(request.url);
    if (pathname === "/1") {
      return new Response(); // ✅
    } else if (pathname === "/2") {
      return new Response("body"); // `TypeError: Can't read from request stream after response has been sent.`
    } else if (pathname === "/3") {
      await request.body.pipeTo(new WritableStream());
      return new Response("body"); // ✅
    }
    return new Response(null, { status: 404 });
  }
}

export default {
  async fetch(request, env, ctx) {
    const id = env.OBJECT.idFromName("");
    const stub = env.OBJECT.get(id);
    return stub.fetch(request);
  }
}

...workerd returns a 200 OK response for all paths, but logs workerd/io/io-context.c++:380: info: uncaught exception; exception = workerd/api/global-scope.c++:81: failed: jsg.TypeError: Can't read from request stream after response has been sent. when --verbose is enabled and /2 is requested with a body (e.g. $ curl -d 0123456789 http://127.0.0.1:8080/2). This only reproduces in a Durable Object, and not when an empty response is returned or the body is consumed.

[jamesgpearce]

Thank you for figuring this out. I thought I was going crazy.

[irvinebroque]

I just tried reproducing this using latest version of workerd and could not, the /2 case as described above does not throw a TypeError. Digging around for commits where we might have addressed, but not closed out this issue.

[mrbbot]

@irvinebroque I was just able to reproduce this on main (85c774c7ba4c498bbb92e97e96198719e83afc32). Make sure you've got the --verbose flag set, and you're sending a request with a body. You should also be able to see the error if you set the --inspector-addr=127.0.0.1:9229 flag, and open the DevTools console from chrome://inspect. 👍

[irvinebroque]

I'm running this via Wrangler, and can't repro: https://github.com/irvinebroque/workerd-918-repro/blob/main/src/worker.ts

[mrbbot]

Hmmm... I seem to be able to repro with your repo... 😅 I assume you're running on arm macOS?

[BrandonRomano]

I also am observing this bug. I am on an ARM MacOS 😄

[dpeek]

I spent some time digging into this today. We were experiencing consistent failiures of certain tests in our API test suite (using ustable_dev and vite). The failures were actually caused by a workerd exception from the test before the failing tests, which I only noticed by running in sequence and turning on debug level logging.

It looks like workerd throws this exception (originating here) any time a request body is not consumed and a response is sent. So, in my case, I was checking auth first and throwing on fail, and then parsing request body. Parsing the body first fixed the failures.

Clients passing request bodies where one isn't expected should (in theory) cause the same exception in production. I guess this is fine because Cloudflare just spins up another worker and gets on with it's life, but this seems like it should be avoided.

Title should be updated to:

"workerd throws Can't read from request stream after response has been sent exception when responding to a request with an unconsumed body"