Why is the SDK listed as a production dependency?

[jdmarshall]

What happened?

Steps to Reproduce

At the start of the docs, the sdk is listed as a dependency.

Expected Result

npm install --save-dev @opentelemetry/sdk-node

Things that are needed at run time are generally not referred to as SDKs.

Actual Result

npm install --save @opentelemetry/sdk-node

Additional Details

The SDK is well over half of the node_modules directory for the given instructions, and when installed into an existing project can lead to module duplication. In a test project, what was 80MB in a fresh directory became over 300MB in a real project.

Is this a documentation issue or is there an unfortunate dependency between the runtime system and the software developer kit that needs to be broken?

OpenTelemetry Setup Code

No response

package.json

No response

Relevant log output

No response

[legendecas]

The term "SDK" used in OpenTelemetry is referring to the implementation of the OpenTelemetry API:

https://opentelemetry.io/docs/concepts/components/#specification

• API: Defines data types and operations for generating and correlating tracing, metrics, and logging data.
• SDK: Defines requirements for a language-specific implementation of the API. Configuration, data processing, and exporting concepts are also defined here.

The package @opentelemetry/sdk-node should consume up to 12M in total in a fresh install. Would you mind providing a reproduction that installing @opentelemetry/sdk-node takes up to 220M?

[dyladan]

Removing the bug label for now. If it turns out we are causing a dependency explosion of some kind maybe it can be reclassified to bug once we know whats happening.

[jdmarshall]

I had some trouble reproducing the exact problem but I have found some clues. The biggest is npm bugs, but the README also triggers some of them. It seems like opentelemetry is fighting with itself on dependency versions, such that an incremental install has a far worse module footprint, which largely comes down to the installation instructions being written for maximum legibility but also triggering this issue.

In a clean directory:

npm install --save @opentelemetry/api
npm install --save @opentelemetry/sdk-node
npm install --save @opentelemetry/auto-instrumentations-node
du -sh node_modules
 81M	node_modules

The above is at the top of the README. The sdk is never implied to be usable on its own. Any example I find includes at least three times the node_modules weight of the sdk.

rm -rf node_modules package-lock.json
npm install
du -sh node_modules
 44M	node_modules

Seems like the open telemetry libraries are directly conflicting with each other over dependencies.

npm install --save @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node
du -sh node_modules
 44M	node_modules/

Meanwhile, in a real project:

du -sh node_modules/
545M	node_modules/
npm install --save @opentelemetry/api
npm install --save @opentelemetry/sdk-node
npm install --save @opentelemetry/auto-instrumentations-node
du -sh node_modules/
632M	node_modules/

The biggest increase though comes from npm bugs.

du -sh node_modules/
569M	node_modules/
npm install --save @opentelemetry/api
npm install --save @opentelemetry/sdk-node
npm install --save @opentelemetry/auto-instrumentations-node
du -sh node_modules/
853M	node_modules/

npm loves to install dev dependencies even on an npm install --save

du -sh node_modules/
580M	node_modules
npm install --save @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node
du -sh node_modules
819M	node_modules
rm -rf node_modules package-lock.json
npm install --prod
du -sh node_modules
619M	node_modules

Accounting for that:

npm install
du -sh node_modules
781M	node_modules/
npm install --save @opentelemetry/api
npm install --save @opentelemetry/sdk-node
npm install --save @opentelemetry/auto-instrumentations-node
du -sh node_modules
866M	node_modules

I do a fair bit of technology selection. There are any number of criteria used to defend a decision, but the first two gates are typically 'how bad is the setup process' and the second is 'how bad is the resource cost'. Those can kill an option before people have even given it a fair chance.

[dyladan]

Thanks for doing the research here. There's not much we can do about npm installing the dev dependencies, but if you're worried about deployment size, then installing individual instrumentations instead of the auto-instrumentations-node package can help a lot as well since it is likely installing a lot of instrumentations you're not using. I would guess that is the vast majority of the cause of the size explosion. Since that package aggregates a bunch of instrumentations which are individually updated and maintained it is very likely that it is installing multiple versions of a lot of the dependencies. We're working on reducing the impact of the instrumentation installation but this is a great data point.

Here are some things we're doing or could do to reduce the contrib/instrumentation footprint:

1. Review contrib modules to see which are not being actively maintained and remove them from auto instrumentations node package.
2. Review contrib modules for outdated dependencies and ensure their dependency versions match so NPM installs as few unnecessary versions of dependencies as possible
3. Remove contrib dependencies on the packages they instrument. This was originally done for type safety and they were included as dev dependencies, but typescript has changed the way dependency types are checked and it was necessary to move some of them to "real" dependencies. We really don't want that so we're working to remove dependence on these types.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions[bot]]

This issue was closed because it has been stale for 14 days with no activity.