Memory Leak When Using opaque Parameter in produce

[patrykwegrzyn]

When calling produce on a Producer and passing any value other than undefined for the opaque parameter results in a memory leak.

Steps to Reproduce

1. Use the following code to produce messages:

   const { Producer } = require("@confluentinc/kafka-javascript");
   const { once } = require("events");
   
   const producerConfig = {
     "client.id": "my-kafka-consumer",
     "metadata.broker.list": "localhost:19092",
   };
   
   const topic = "test-topic";
   const producer = new Producer(producerConfig);
   
   function getRandomInt(min, max) {
     const ceilMin = Math.ceil(min);
     const floorMax = Math.floor(max);
     return Math.floor(Math.random() * (floorMax - ceilMin + 1)) + ceilMin;
   }
   
   function startDataGeneration() {
     producer.setPollInterval(10);
     const intervalMs = 1;
   
     setInterval(() => {
       for (let i = 0; i < 100; i++) {
         const payload = {
           timestamp: new Date().toISOString(),
           message: "Hello from data generator!",
           random: getRandomInt(0, 999),
           index: i,
         };
   
         const value = Buffer.from(JSON.stringify(payload));
         const key = `key-${getRandomInt(1, 20)}`;
   
         producer.produce(topic, null, value, key, null, { a: 1 });
       }
     }, intervalMs);
   }
   
   async function main() {
     producer.connect();
     await once(producer, "ready");
     console.log("Producer is ready. Starting data generation...");
     startDataGeneration();
   }
   
   main().catch((err) => {
     console.error("Error in main execution:", err);
   });
   

2. Run the above script and wait for at least 5 minutes.
3. Observe that memory usage continues to grow indefinitely.
4. Replace { a: 1 } with undefined in the opaque parameter.
5. Notice that the memory leak does not occur when opaque is undefined.

Expected Behaviour

Passing an object to opaque should not cause a memory leak.

Actual Behaviour

Memory usage continues to grow indefinitely when opaque is set to { a: 1 }, null, or any value other than undefined.

Environment

• @confluentinc/kafka-javascript version: v1.2.0
• Node.js version: 22.4
• OS: Arch Linux

Additional Context

• The issue persists across multiple runs.
• Removing opaque (or setting it to undefined) prevents the memory leak.

[patrykwegrzyn]

adding dr_cb: true seems to be fixing the issue

[milindl]

Thanks for filing this. There's a bug. We create an JS object for opaque from C++, which is deleted after the delivery report cb is called. If there is no registered delivery callback, we don't end up deleting this. The solution would be to not create the JS object in the first place if there's no delivery cb supplied.