berry icon indicating copy to clipboard operation
berry copied to clipboard

[Bug?]: yarn workspaces focus does not default to --immutable on CI

Open ranisalt opened this issue 1 year ago โ€ข 6 comments

Self-service

  • [x] I'd be willing to implement a fix

Describe the bug

When running yarn install on CI (GitHub Actions in my specific scenario), it will enable --immutable and prevent continuing if the install happens to update the lockfile. However, then focus installing - which we do to speed up installing dependencies on CI - it does not. There is no --immutable flag to workspaces focus yet for me to manually enable it either.

Running yarn install --mode=update-lockfile as a separate step works but I would like to avoid the extra step, since install already has that behavior and workspaces focus should follow suit.

To reproduce

package.json:

{
  "name": "yarn-bug",
  "packageManager": "[email protected]",
  "dependencies": {
    "lodash": "4.17.21"
  }
}

yarn.lock:

# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!

__metadata:
  version: 8
  cacheKey: 10c0

"lodash@npm:4.17.20":
  version: 4.17.20
  resolution: "lodash@npm:4.17.20"
  checksum: 10c0/faec37cb9f161b766bdc078a1356a07b9eaaa867796dd2520a407fe0a6a6d7be031e8f228f0cf3d305095703ee40258616c870b8d17dcdcb16f745bf31e8c3c2
  languageName: node
  linkType: hard

"yarn-bug@workspace:.":
  version: 0.0.0-use.local
  resolution: "yarn-bug@workspace:."
  dependencies:
    lodash: "npm:4.17.20"
  languageName: unknown
  linkType: soft

Running yarn install --immutable (or running in CI) results in the following:

โžค YN0000: ยท Yarn 4.1.1
โžค YN0000: โ”Œ Resolution step
โžค YN0085: โ”‚ + lodash@npm:4.17.21
โžค YN0085: โ”‚ - lodash@npm:4.17.20
โžค YN0000: โ”” Completed

โžค YN0000: โ”Œ Post-resolution validation
โžค YN0000: โ”‚ @@ -4,18 +4,17 @@
โžค YN0000: โ”‚  __metadata:
โžค YN0000: โ”‚    version: 8
โžค YN0000: โ”‚    cacheKey: 10c0
โžค YN0000: โ”‚
โžค YN0028: โ”‚ -"lodash@npm:4.17.20":
โžค YN0028: โ”‚ -  version: 4.17.20
โžค YN0028: โ”‚ -  resolution: "lodash@npm:4.17.20"
โžค YN0028: โ”‚ -  checksum: 10c0/faec37cb9f161b766bdc078a1356a07b9eaaa867796dd2520a407fe0a6a6d7be031e8f228f0cf3d305095703ee40258616c870b8d17dcdcb16f745bf31e8c3c2
โžค YN0028: โ”‚ +"lodash@npm:4.17.21":
โžค YN0028: โ”‚ +  version: 4.17.21
โžค YN0028: โ”‚ +  resolution: "lodash@npm:4.17.21"
โžค YN0000: โ”‚    languageName: node
โžค YN0000: โ”‚    linkType: hard
โžค YN0000: โ”‚
โžค YN0000: โ”‚  "yarn-bug@workspace:.":
โžค YN0000: โ”‚    version: 0.0.0-use.local
โžค YN0000: โ”‚    resolution: "yarn-bug@workspace:."
โžค YN0000: โ”‚    dependencies:
โžค YN0028: โ”‚ -    lodash: "npm:4.17.20"
โžค YN0028: โ”‚ +    lodash: "npm:4.17.21"
โžค YN0000: โ”‚    languageName: unknown
โžค YN0000: โ”‚    linkType: soft
โžค YN0000: โ”‚
โžค YN0028: โ”‚ The lockfile would have been modified by this install, which is explicitly forbidden.
โžค YN0000: โ”” Completed
โžค YN0000: ยท Failed with errors in 0s 22ms

It is not possible to run yarn workspaces focus --immutable and in CI it silently updates the lockfile.

Environment

System:
    OS: macOS 14.4.1
    CPU: (10) arm64 Apple M1 Pro
  Binaries:
    Node: 20.11.1 - /private/var/folders/03/j2jymg9n377bdxjh26rpckv40000gp/T/xfs-ee1b3860/node
    Yarn: 4.1.1 - /private/var/folders/03/j2jymg9n377bdxjh26rpckv40000gp/T/xfs-ee1b3860/yarn
    npm: 10.2.4 - ~/.config/asdf/plugins/nodejs/shims/npm

Additional context

The link to sherlock documentation is broken

ranisalt avatar May 06 '24 13:05 ranisalt

I'm also encountering this (or something really similar), and am not sure how we should proceed.

Edit: I downgraded to yarn 3 and no longer have this problem, not sure how to fix it with yarn 4

andreieftimie avatar Jun 27 '24 07:06 andreieftimie

The --immutable flag would make little sense on focus since it doesn't install your whole project, so it has no idea whether the lockfile would need to be updated or not.

At best it could tell you whether a specific part of your projects would need to be updated, but since it could only check whether something needs to be added (and not whether something needs to be removed) you'd still need to run a proper full immutable install somewhere, so this wouldn't be useful.

arcanis avatar Jun 27 '24 10:06 arcanis

Hi! ๐Ÿ‘‹

It seems like this issue as been marked as probably resolved, or missing important information blocking its progression. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it.

yarnbot avatar Jul 27 '24 10:07 yarnbot

At best it could tell you whether a specific part of your projects would need to be updated, but since it could only check whether something needs to be added (and not whether something needs to be removed) you'd still need to run a proper full immutable install somewhere, so this wouldn't be useful.

I don't get it. Here's what I want to do:

The --immutable flag would make little sense on focus since it doesn't install your whole project, so it has no idea whether the lockfile would need to be updated or not.

If the focus install would require the lockfile to be updated, for example if a dependency version is changed without a reinstall (i.e. people working in a focused install instead of a complete install) then this would detect the issue in CI

My use case is that we have multiple teams working in a monorepo and it's not convenient that everyone has a full install at all times

ranisalt avatar Jul 27 '24 18:07 ranisalt

If the focus install would require the lockfile to be updated...

A focused install, almost by definition, would require the lockfile to be updated. A focused install literally mutates the in-memory data structures like parsed package.jsons, then runs a normal install. That would produce a different lockfile, we just don't write that back to disk.

Thus, an immutable focused install either

  • (almost) always throws; or
  • needs to fetch all packages to check for immutability, which defeats the purpose of a focused install

So you don't want an immutable focused install, you want a focused install where some mutations are okay and other are not. The question then becomes what mutations are okay and how to determine if a given change is okay or not. There just aren't clear-cut answers to those questions that doesn't devolve into one of the two bad options above.

clemyan avatar Jul 29 '24 06:07 clemyan

Hi! ๐Ÿ‘‹

It seems like this issue as been marked as probably resolved, or missing important information blocking its progression. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it.

yarnbot avatar Aug 28 '24 06:08 yarnbot