🐛 BUG: Producer sending message to queue throwing Error: Network connection lost in Miniflare

[Soviut]

Which Cloudflare product(s) does this pertain to?

Miniflare

What version(s) of the tool(s) are you using?

3.20240512.0 [miniflare]

What version of Node are you using?

20.9.0

What operating system and version are you using?

Unbutu 22 under WSL2 on Windows 11

Describe the Bug

My producer can't send messages to my queue in miniflare.

Observed behavior

I'm testing a basic miniflare configuration with two workers and a queue. One worker is the producer, the other is the consumer.

Trying to send any messages to the queue result in the following error.

Error: Network connection lost.

It happens when you call the producer's route (which sends a message to the queue)

const res = await mf.dispatchFetch('http://localhost:5000/producer')

And it also happens when you get the producer directly from the miniflare instance and try to send a message.

const queue = await mf.getQueueProducer('CANDIDATES_QUEUE', 'producer')
queue.send({ ding: 'Hello from Miniflare!' })

Expected behavior

I expect the producer to be able to send a message to the queue and return a response. I expect the consumer to consume the message from the queue and log out what it found.

Steps to reproduce

run the following code with npx tsx .

import { Miniflare } from 'miniflare'

const mf = new Miniflare({
  // host: '0.0.0.0',
  port: 5000,

  workers: [
    {
      name: 'producer',
      modules: true,
      routes: ['http://localhost/producer*'],
      script: `
        export default {
          async fetch(req, env, ctx) {
            console.log('fetch')
            await env.CANDIDATES_QUEUE.send({
              testing: 'Hello from Miniflare!',
            })
            return new Response('Producer sent message to queue');
          }
        }
      `,

      queueProducers: {
        CANDIDATES_QUEUE: 'candidates-queue',
      },
    },
    {
      name: 'consumer',
      modules: true,
      routes: ['http://localhost/queue*'],
      script: `
        export default {
          async queue(batch, env) {
            console.log('received ' + batch.messages.length)
            for (let message of batch.messages) {
              console.log(
                'message ' + message.id + ' processed: ' + JSON.stringify(message.body)
              )
            }
          }
        }
      `,

      queueConsumers: ['candidates-queue'],
    },
  ],
})

// this throws the network error
const res = await mf.dispatchFetch('http://localhost:5000/producer')
console.log(res.text())

// this throws the network error
const queue = await mf.getQueueProducer('CANDIDATES_QUEUE', 'producer')
queue.send({ ding: 'Hello from Miniflare!' })

Please provide a link to a minimal reproduction

No response

Please provide any relevant error logs

node:internal/process/task_queues:95
    runMicrotasks();
    ^

Error: Network connection lost.
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  [cause]: undefined
}

Node.js v20.9.0

[TimoWilhelm]

Seeing the same issues on Windows 11 Node v21.7.3 with miniflare 3.20240610.1. All queue producers fail with Error: Network connection lost.

[TimoWilhelm]

I assume this issue is caused by https://github.com/cloudflare/workers-sdk/commit/66bdad08834b403100d1e4d6cd507978cc50eaba.

The zod schema for queueProducers is wrong and the Record<string, string> format is no longer valid. Using Record<string, QueueProducerOptions> works.

queueProducers: {
  CANDIDATES_QUEUE: {
    queueName: 'candidates-queue',
  },
},

I created a PR to fix the zod schema here https://github.com/cloudflare/workers-sdk/pull/6128.

[j-white]

I encountered this one as well - on a Mac M1 it manifested itself with *** Received signal #11: Segmentation fault: 11 errors instead. This went away after changing the format of the entries in the queueProducers map ¯_(ツ)_/¯

[CarmenPopoviciu]

hi folks,

as @TimoWilhelm correctly pointed out, it would seem that we introduced a regression in https://github.com/cloudflare/workers-sdk/commit/66bdad08834b403100d1e4d6cd507978cc50eaba, namely that declaring Queue Producers like queueProducers: { "MY_QUEUE": "my-queue" }, no longer works. I don't have context of those changes personally, so I am not entirely sure if not supporting that syntax was intentional or not.

If this was intentional, I will land Timo's PR and update the Miniflare docs. If this is indeed a regression, I'll put up a fix asap.

Let me first check with our team the context of those changes, and will circle back here later today.

[CarmenPopoviciu]

can confirm that indeed we introduced a regression for use cases where the queue producer is specified as

queueProducers: {
       QUEUE: 'my-queue',
 },

I have a PR up that will fix that. I am not sure however if that will fix the issue described in this bug report. Once the fix PR lands, I'll circle back here to check if you are still experiencing issues.

[CarmenPopoviciu]

hey folks,

my (hopefully) fix PR was merged and released today. Can someone please confirm whether you are still experiencing this issue on latest wrangler version 3.78.5 🙏