Support instrumentation of modules via bundler plugins

[drewcorlin1]

trafficstars

Is your feature request related to a problem? Please describe.

I am attempting to use the OTel auto instrumentation for Javascript, but non-built-in modules are not patched because I am bundling my code with esbuild before deploying it into a docker container (same scenario as https://github.com/open-telemetry/opentelemetry-js/issues/2708).

Describe the solution you'd like

Provide bundler (esbuild, webpack, rollup, etc.) plugins as a means of instrumenting third party modules, since the existing require()-based patching does not work if you are bundling your code. This is somewhat of an alternative to https://github.com/open-telemetry/opentelemetry-js/issues/2708, with a big difference I see being that the bundler plugin could more easily apply to modules only when they're present. The proposed solution in that link seems much more manual to maintain in a codebase where we have many microservices with varying dependencies. I don't want to force developers to know they need to pass 20 aws modules to the OTel instrumentation just because they added an aws-s3 dependency.

Describe alternatives you've considered

1. https://github.com/open-telemetry/opentelemetry-js/issues/2708
2. Writing this esbuild plugin myself (code below).

Additional context

An extremely hacky proof of concept with esbuild's onLoad hook to patch pino and fastify

// esbuild.ts
import { build, PluginBuild } from 'esbuild';
import { readFile } from 'fs/promises';

function wrapModule({
  originalSource,
  instrumentationPackage,
  instrumentationName,
  instrumentationConstructorArgs,
}: {
  originalSource: string;
  instrumentationPackage: string;
  instrumentationName: string;
  instrumentationConstructorArgs?: string;
}) {
  return `
(function() {
${originalSource}
})(...arguments);
{
let mod = module.exports;

const { ${instrumentationName} } = require('${instrumentationPackage}');
const instrumentations = new ${instrumentationName}(${instrumentationConstructorArgs ?? ''}).init();
// TODO: Get rid of this check, but also ensure it does what we want when there are multiple instrumentations
if (instrumentations.length > 1) throw new Error('Cannot handle multiple instrumentations');

for (const instrumentation of instrumentations) {
        mod = instrumentation.patch(mod);
}
module.exports = mod;
}
`;
}

function loadFastify(build: PluginBuild) {
  build.onLoad({ filter: /fastify\/fastify.js$/ }, async () => {
    const resolved = await build.resolve('./fastify', {
      kind: 'require-call',
      resolveDir: './node_modules',
    });

    const contents = await readFile(resolved.path);
    return {
      contents: wrapModule({
        originalSource: contents.toString(),
        instrumentationPackage: '@opentelemetry/instrumentation-fastify',
        instrumentationName: 'FastifyInstrumentation',
      }),
      resolveDir: './node_modules/fastify',
    };
  });
}

interface PinoConfig {
  // TODO: Improve types
  logHook: (span: any, record: any) => void;
}

function loadPino(build: PluginBuild, config?: PinoConfig) {
  build.onLoad({ filter: /pino\/pino.js$/ }, async () => {
    const resolved = await build.resolve('./pino', {
      kind: 'require-call',
      resolveDir: './node_modules',
    });

    const contents = await readFile(resolved.path);
    return {
      contents: wrapModule({
        originalSource: contents.toString(),
        instrumentationPackage: '@opentelemetry/instrumentation-pino',
        instrumentationName: 'PinoInstrumentation',
        instrumentationConstructorArgs: `{
          logHook: ${config?.logHook.toString() ?? undefined},
        }`,
      }),
      resolveDir: './node_modules/pino',
    };
  });
}

build({
  entryPoints: ['src/server.ts'],
  bundle: true,
  outfile: 'dist/server.js',
  target: 'node18',
  platform: 'node',
  sourcemap: true,
  plugins: [
    {
      name: 'open-telemetry',
      setup(build) {
        loadFastify(build);
        loadPino(build, {
          logHook: (span, record) => {
            // Reformat the injected log fields to use camelCase, eg. trace_id -> traceId
            const context = span.spanContext();
            record.traceId = context.traceId;
            record.spanId = context.spanId;
            record.strTraceFlags = context.traceFlags;

            if (record.trace_id === context.traceId) delete record.trace_id;
            if (record.span_id === context.spanId) delete record.span_id;
            if (Number(record.trace_flags) === context.traceFlags) delete record.trace_flags;
          },
        });
      },
    },
  ],
}).catch(err => {
  throw err;
});

[drewcorlin1]

DataDog has an experimental esbuild plugin and I think a similar approach could be used for this. Is there any interest in this solution? I would be happy to begin contributing for a few packages, but I don't think I could commit to getting the plugin to work for every supported package.

[mrjackdavis]

Hey! I just stumped upon this and it'd be extremely helpful if there was a community solution for this @drewcorlin1

You have my +1

[RichiCoder1]

This would be an awesome way to approach this. It looks like DD's work is open source too, might be able to reference their work or collab: https://github.com/DataDog/dd-trace-js/blob/master/packages/datadog-esbuild/index.js

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[drewcorlin1]

Not stale

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[drewcorlin1]

being addressed here https://github.com/open-telemetry/opentelemetry-js-contrib/pull/1856