confluent-kafka-javascript icon indicating copy to clipboard operation
confluent-kafka-javascript copied to clipboard

Error instantiating a Kafka client with the Bun runtime

Open arcoraven opened this issue 10 months ago • 4 comments

Environment Information

  • OS: MacOS 15.2, Linux x64
  • Bun version: 1.2.3 and 1.0.0
  • confluent-kafka-javascript version: 1.2.0 and 1.0.0

Steps to Reproduce Trying to instantiate the KafkaJS client with the Bun runtime throws an error.

index.ts:

import { KafkaJS } from "@confluentinc/kafka-javascript";
new KafkaJS.Kafka({});

Running it with bun run index.ts:

$ bun run index.ts

107 |         return opts[p] || p;
108 |       })
109 |     );
110 |     tries.push(n);
111 |     try {
112 |       b = opts.path ? requireFunc.resolve(n) : requireFunc(n);
                                                     ^
error: The module 'kafka' was compiled against a different Node.js ABI version using NODE_MODULE_VERSION 108. This version of Bun requires NODE_MODULE_VERSION 127. Please try re-compiling or re-installing the module.
      at bindings (/Users/arcoraven/bun-example/node_modules/bindings/bindings.js:112:48)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/librdkafka.js:10:5)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/client.js:15:5)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/kafka-consumer.js:14:5)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/rdkafka.js:11:5)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/kafkajs/_producer.js:1:7)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/kafkajs/_kafka.js:1:19)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/kafkajs/index.js:5:8)
      at <anonymous> (/Users/arcoraven/bun-example/node_modules/@confluentinc/kafka-javascript/lib/index.js:1:5)

Bun v1.2.3 (macOS arm64)

Does this library depend on Node.js-specific APIs or V8 engine features that aren't yet implemented in Bun's runtime?

confluent-kafka-javascript Configuration Settings N/A

Additional context

package.json:

{
  "name": "bun-example",
  "module": "index.ts",
  "type": "module",
  "private": true,
  "devDependencies": {
    "@types/bun": "latest"
  },
  "peerDependencies": {
    "typescript": "^5"
  },
  "dependencies": {
    "@confluentinc/kafka-javascript": "1.2.0"
  },
  "trustedDependencies": [
    "@confluentinc/kafka-javascript"
  ]
}

tsconfig.json:

{
  "compilerOptions": {
    // Enable latest features
    "lib": ["ESNext", "DOM"],
    "target": "ESNext",
    "module": "ESNext",
    "moduleDetection": "force",
    "jsx": "react-jsx",
    "allowJs": true,

    // Bundler mode
    "moduleResolution": "bundler",
    "allowImportingTsExtensions": true,
    "verbatimModuleSyntax": true,
    "noEmit": true,

    // Best practices
    "strict": true,
    "skipLibCheck": true,
    "noFallthroughCasesInSwitch": true,

    // Some stricter flags (disabled by default)
    "noUnusedLocals": false,
    "noUnusedParameters": false,
    "noPropertyAccessFromIndexSignature": false
  }
}

arcoraven avatar Feb 26 '25 13:02 arcoraven

I get slightly different error bun: symbol lookup error: node_modules/@confluentinc/kafka-javascript/build/Release/confluent-kafka-javascript.node: undefined symbol: _ZN2v816FunctionTemplate12SetClassNameENS_5LocalINS_6StringEEE

I can see similar for node-rdkafka and others

https://github.com/oven-sh/bun/issues/15923 https://github.com/oven-sh/bun/issues/4290

I guess this one is on bun side to resolve?

patrykwegrzyn avatar Feb 26 '25 23:02 patrykwegrzyn

To get bun support, either they'll have to support all the nan functions we use (What @patrykwegrzyn has linked above), or else, we will need to transition to N-API which is better supported by bun (https://github.com/oven-sh/bun/issues/158).

However, I think both these alternatives will need a fair bit of work testing and making sure everything works properly. So, at the moment, we don't support bun (or any runtime except node itself).

milindl avatar Feb 27 '25 04:02 milindl

I tried using this today with Deno with a similar result. (In this case the node_module_register() symbol was missing.

By supporting N-API it would gain support for both Deno and Bun

cowboyd avatar Mar 19 '25 21:03 cowboyd

Here is a draft of the migration https://github.com/confluentinc/confluent-kafka-javascript/pull/281

cowboyd avatar Mar 27 '25 04:03 cowboyd