confluent-kafka-javascript icon indicating copy to clipboard operation
confluent-kafka-javascript copied to clipboard

Unable to override binary download host

Open sdgrimm opened this issue 9 months ago • 1 comments

I need to override the binary download host for offline installs. This is necessary in order to use this package in airgapped environments not connected to the internet. node-pre-gyp suggests this is supported but it does not work. I found a bug with how it replaces dashes: https://github.com/mapbox/node-pre-gyp/issues/896

Just a suggestion for an alternative to managing supported binaries... some packages use the approach of packaging supported binaries into npm packages listed as optional dependencies with the appropriate os defined. For example, esbuild does this here. This provides a really nice experience for running airgapped environments because nothing needs to be overridden and the npm packages are simply proxied/replicated as is.

sdgrimm avatar Mar 18 '25 15:03 sdgrimm

I don't know of a definite way to set this up - the idea of creating optional dependencies for each of our arch,os,node-version,libc versions is something we can consider, after working out the operational stuff around it (we'll need around 35-40 'shell' packages).

In the meanwhile I can suggest a workaround, and that's to download dependencies manually before taking your folder to the airgapped environment, if that's possible.

# on non-airgapped env:
wget https://github.com/confluentinc/confluent-kafka-javascript/releases/download/v1.4.0/confluent-kafka-javascript-v1.4.0-node-v120-linux-glibc-x64.tar.gz

# on airgapped env:
npm ci --ignore-scripts # this won't run node-pre-gyp and won't call external urls
cp /tmp/some/pre_copied_location/confluent-kafka-javascript-v1.4.0-node-v120-linux-glibc-x64.tar.gz .
tar xvf confluent-kafka-javascript-v1.4.0-node-v120-linux-glibc-x64.tar.gz
mv Release node_modules/@confluentinc/kafka-javascript/

In this case, you would need to figure out the relevant details in the download URL: it has node abi version (it's fixed for every major version of node, you can use the node-abi package on any machine to get this info), the os, the libc version (only for linux), and the architecture. A listing of all available binaries is available on the Releases page.

milindl avatar Jul 09 '25 10:07 milindl