turbo icon indicating copy to clipboard operation
turbo copied to clipboard

Turbo prune does not respect yarn v1 lock file protocols split entries

Open Netail opened this issue 1 year ago • 3 comments

Verify canary release

  • [X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/Netail/repro-yarn-lock

What package manager are you using / does the bug impact?

Yarn v1

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.0.11

Describe the Bug

Pruned yarn.lock results in merging npm: prefixed versions & non-prefixed versions

"string-width-cjs@npm:string-width@^4.2.0", [email protected], string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
  version "4.2.3"
  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
  dependencies:
    emoji-regex "^8.0.0"
    is-fullwidth-code-point "^3.0.0"
    strip-ansi "^6.0.1"

Expected Behavior

Split yarn.lock entries starting with the npm: prefix

"string-width-cjs@npm:string-width@^4.2.0":
  version "4.2.3"
  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
  dependencies:
    emoji-regex "^8.0.0"
    is-fullwidth-code-point "^3.0.0"
    strip-ansi "^6.0.1"

string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
  version "4.2.3"
  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
  dependencies:
    emoji-regex "^8.0.0"
    is-fullwidth-code-point "^3.0.0"
    strip-ansi "^6.0.1"

To Reproduce

  1. Have the split yarn.lock entries in your original yarn.lock
  2. Prune repo
  3. Check pruned yarn.lock
  4. See that they are merged again

Additional context

This issue got fixed in Yarn 1.22.22 with https://github.com/yarnpkg/yarn/pull/9023.

A full thread on the issue; https://github.com/yargs/cliui/issues/138

Netail avatar Jul 26 '24 11:07 Netail

Important parts in the repro;

package.json (versions defined) https://github.com/Netail/repro-yarn-lock/blob/main/apps/web/package.json#L5-L8

yarn.lock (how the yarn.lock entries also should look like after being pruned) https://github.com/Netail/repro-yarn-lock/blob/main/yarn.lock#L25-L41

Currently blocking us from optimally using prune in our CI as we have to delete the pruned yarn.lock and generate a new yarn.lock based on the pruned repo. Slowing the pipeline a bit :(

Netail avatar Aug 07 '24 21:08 Netail

The same for theyarn:, github:, git protocols. (And file:, portal: & link:, but those are not even copied over, related to: https://github.com/vercel/turborepo/issues/4105).

Basically all version fields starting with a protocol prefix should end up as separate entries in the yarn.lock

Netail avatar Sep 20 '24 13:09 Netail

@chris-olszewski are you able to help me with this issue? yarn.lock entries with a protocol shouldn't be merged together (neither with one without a protocol) Probably something with; https://github.com/vercel/turborepo/blob/6d4e655b0120c96e20308adbf46380e5b71a7db1/crates/turborepo-lockfiles/src/yarn1/mod.rs#L148-L155

Where it tries to check if anything matches with a protocol

Netail avatar Sep 25 '24 14:09 Netail

@Netail Sorry for the delay on fixing this. This ended up being an issue just with our implementation of Yarn's serialization logic not matching a post https://github.com/yarnpkg/yarn/pull/9023/ yarn version. #9627 should provide the fix for this by no longer collapsing entries for descriptors that have different package names.

chris-olszewski avatar Dec 17 '24 23:12 chris-olszewski

Awesome, thanks! No worries on the delay. Tried to take a stab at it too with https://github.com/vercel/turborepo/pull/9349, but apparently that was not the issue xd

Netail avatar Dec 17 '24 23:12 Netail