`bun install` routinely accumulates package duplication

[brettwillis]

What version of Bun is running?

1.1.26

What platform is your computer?

Darwin 23.6.0 arm64 arm

What steps can reproduce the bug?

When updating package versions, either manually in the package.json and running bun install, or by running bun update, Bun routinely starts duplicating packages in the dependency tree and then continues to accumulate duplicated dependencies over time, entirely un-necessarily.

This happens silently until we realise what has been happening due to obscure runtime bugs or TypeScript type checking errors.

Now, we always need to blast away the entire installation and lockfile and install from scratch every time to avoid these obscure problems.

Reproduction:

Consider then packages

• @myorg/[email protected]
• @myorg/[email protected] depends on @myorg/dep1@^1.0.89

When installing, we get a good tree like this:

% bun pm ls --all
./myproject/core node_modules
├── @myorg/[email protected]
├── @myorg/[email protected]

Now, @myorg/dep1 is updated to 1.0.90, and we update the dependency, and the tree becomes:

% bun pm ls --all
./myproject/core node_modules
├── @myorg/[email protected]
├── @myorg/[email protected]
│   └── @myorg/[email protected]

This prolific duplication seems crazy.

What is the expected behavior?

I expect Bun should only duplicate the dependency tree when absolutely necessary to maintain semver compatibility. Dependencies should always be deduplicated/hoisted except where the hoisted version cannot meet semver requirements.

TL;DR prioritise having a single version of each dependency.

What do you see instead?

See above.

Additional information

npm update

npm will prioritize having a single version of [dependencies] in your tree rather than two when that single version can satisfy the semver requirements of multiple dependencies in your tree.

npm install

hoisted (default): Install non-duplicated in top-level, and duplicated as necessary within directory structure

[Jarred-Sumner]

Can you paste a package.json + bun.lockb with a more clear example?

[brettwillis]

Sure, note these are private org dependencies, but hopefully the info can tell you what you need.

Step 1

package.json

{
  "dependencies": {
    "@arrowheadapps/billing-engine-core": "1.0.89",
    "@arrowheadapps/billing-engine-platform-core": "1.0.89"
  }
}

bun.lockb

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
# bun ./bun.lockb --hash: 4677EDE48AFE075F-9887327d4f9c38db-9FB4A885D13A430A-aba16b62138a868f
 
 
"@arrowheadapps/[email protected]", "@arrowheadapps/billing-engine-core@^1.0.88": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-core/1.0.89/b7f345abb57b4ba15cc5a6ef475f2b85d01ad0f4"
  integrity sha512-24l6Z/5s18UdHSVysjmjFitFalfBdXGM7VQ9WchImq6Bh3/q/UDjDp/1fBmRrplsL1cytKbVYBHDpkQxURKdaw==
  dependencies: 
    date-fns "^3.6.0"
    date-fns-tz "^3.1.3"
 
"@arrowheadapps/[email protected]": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-platform-core/1.0.89/c98cfd0745e018887356f4b33a84f9dd7da51369"
  integrity sha512-5Cv+VWHJI951sCIiPQs1rYww8VEbv7LrU//CtFivTEMq/WWEgXzmJSMkbxRQCPs8mD16BVgG6cpLyUpt1nYUeA==
  dependencies: 
    "@arrowheadapps/billing-engine-core" "^1.0.88"
    "@arrowheadapps/database" "^1.24.0"
 
"@arrowheadapps/database@^1.24.0": 
  version "1.24.0"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/database/1.24.0/d8c5e785abada06c63015a71ac75477099564e06"
  integrity sha512-rfK4rT5SyDBZVly/8xrAnekHsGFbrhNRYoOi+Ehe1BEm9D3iMS3aQBD+zKZz3GaxcimqhSKdMJCezZpwkKvQLA==
 
date-fns@^3.0.0, date-fns@^3.6.0: 
  version "3.6.0"
  resolved "https://registry.npmjs.org/date-fns/-/date-fns-3.6.0.tgz"
  integrity sha512-fRHTG8g/Gif+kSh50gaGEdToemgfj74aRX3swtiouboip5JDLAyDE9F11nHMIcvOaXeOC6D7SpNhi7uFyB7Uww==
 
date-fns-tz@^3.1.3: 
  version "3.1.3"
  resolved "https://registry.npmjs.org/date-fns-tz/-/date-fns-tz-3.1.3.tgz"
  integrity sha512-ZfbMu+nbzW0mEzC8VZrLiSWvUIaI3aRHeq33mTe7Y38UctKukgqPR4nTDwcwS4d64Gf8GghnVsroBuMY3eiTeA==

Step 2

Changed versions in package.json to allow upgrading, ran bun update.

package.json

{
  "dependencies": {
    "@arrowheadapps/billing-engine-core": "^1.0.90",
    "@arrowheadapps/billing-engine-platform-core": "^1.0.89"
  }
}

Edit: sorry I originally didn't refresh the view of the lockfile, updated with the correct content:

bun.lockb

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
# bun ./bun.lockb --hash: BAEA39DBE85E244A-590f4688d801b0ed-46555B4DF4C22FF4-aa6ebccac756b473
 
 
"@arrowheadapps/billing-engine-core@^1.0.88": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-core/1.0.89/b7f345abb57b4ba15cc5a6ef475f2b85d01ad0f4"
  integrity sha512-24l6Z/5s18UdHSVysjmjFitFalfBdXGM7VQ9WchImq6Bh3/q/UDjDp/1fBmRrplsL1cytKbVYBHDpkQxURKdaw==
  dependencies: 
    date-fns "^3.6.0"
    date-fns-tz "^3.1.3"
 
"@arrowheadapps/billing-engine-core@^1.0.89": 
  version "1.0.90"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-core/1.0.90/32e97822444abfc3a479959d7ce21460a75a0899"
  integrity sha512-hkBFsAzN1yHyKRPN3yYDqypoFKuslomTdaNy3ixaXRQ8nHIgGgARa3jZZhrx2a+LY97mKoQYNF+OVZgO1bnjZg==
  dependencies: 
    date-fns "^3.6.0"
    date-fns-tz "^3.1.3"
 
"@arrowheadapps/billing-engine-platform-core@^1.0.89": 
  version "1.0.89"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/billing-engine-platform-core/1.0.89/c98cfd0745e018887356f4b33a84f9dd7da51369"
  integrity sha512-5Cv+VWHJI951sCIiPQs1rYww8VEbv7LrU//CtFivTEMq/WWEgXzmJSMkbxRQCPs8mD16BVgG6cpLyUpt1nYUeA==
  dependencies: 
    "@arrowheadapps/billing-engine-core" "^1.0.88"
    "@arrowheadapps/database" "^1.24.0"
 
"@arrowheadapps/database@^1.24.0": 
  version "1.24.0"
  resolved "https://npm.pkg.github.com/download/@arrowheadapps/database/1.24.0/d8c5e785abada06c63015a71ac75477099564e06"
  integrity sha512-rfK4rT5SyDBZVly/8xrAnekHsGFbrhNRYoOi+Ehe1BEm9D3iMS3aQBD+zKZz3GaxcimqhSKdMJCezZpwkKvQLA==
 
date-fns@^3.0.0, date-fns@^3.6.0: 
  version "3.6.0"
  resolved "https://registry.npmjs.org/date-fns/-/date-fns-3.6.0.tgz"
  integrity sha512-fRHTG8g/Gif+kSh50gaGEdToemgfj74aRX3swtiouboip5JDLAyDE9F11nHMIcvOaXeOC6D7SpNhi7uFyB7Uww==
 
date-fns-tz@^3.1.3: 
  version "3.1.3"
  resolved "https://registry.npmjs.org/date-fns-tz/-/date-fns-tz-3.1.3.tgz"
  integrity sha512-ZfbMu+nbzW0mEzC8VZrLiSWvUIaI3aRHeq33mTe7Y38UctKukgqPR4nTDwcwS4d64Gf8GghnVsroBuMY3eiTeA==

And output of bun pm ls --all (yes it does correctly reflect the filesystem):

% bun pm ls --all
./test node_modules
├── @arrowheadapps/[email protected]
├── @arrowheadapps/[email protected]
│   └── @arrowheadapps/[email protected]
├── @arrowheadapps/[email protected]
├── [email protected]
└── [email protected]

[Jarred-Sumner]

Thanks. @dylan-conway is able to reproduce it