[Bug]: Yarn packs git dependencies using an incorrect Yarn version

[colbymelvin]

Self-service

• [ ] I'd be willing to implement a fix

Describe the bug

Hi there,

Wanted get some clarity on an issue that my org is currently seeing while attempting to migrate from npm to yarn. After getting everything up and running locally with PnP (pretty easy, nice docs!), we're seeing a 100% build failure on yarn install in the specific feature branch on CI.

The exception thrown is YN0028, which I understand is a result of CI builds being immutable. The question I'm trying to answer is.. Why does CI think the lockfile needs to be updated in the first place?

I've committed yarn.lock to our VCS. Our local environments use the same Docker images as our CI environments, so there shouldn't be any discrepancies there.

Here's a small sample of a diff:

➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ -yocto-queue@^0.1.0:
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ -  version "0.1.0"
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ -  resolved "https://registry.yarnpkg.com/yocto-queue/-/yocto-queue-0.1.0.tgz#0294eb3dee05028d31ee1a5fa2c556a6aaf10a1b"
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ -  integrity sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q==
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ +"yocto-queue@npm:^0.1.0":
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ +  version: 0.1.0
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ +  resolution: "yocto-queue@npm:0.1.0"
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ +  languageName: node
➤ YN0000: │ /tmp/xfs-2aa69eb9 STDOUT ➤ YN0028: │ +  linkType: hard

As you can see, yarn seems to think that the yocto-queue entry needs to be updated. Here's the corresponding entry in our yarn.lock file in our VCS:

"yocto-queue@npm:^0.1.0":
  version: 0.1.0
  resolution: "yocto-queue@npm:0.1.0"
  checksum: f77b3d8d00310def622123df93d4ee654fc6a0096182af8bd60679ddcdfb3474c56c6c7190817c84a2785648cdee9d721c0154eb45698c62176c322fb46fc700
  languageName: node
  linkType: hard

So far I've tried the following locally with no luck:

1. rm -rf .yarn/cache/
2. rm -rf .yarn/unplugged/
3. yarn install

Which does not produce any changes in yarn.lock locally. Doing the above but replacing yarn install with yarn install --immutable does not error locally, either (I would have expected it to since the same command in CI fails).

I guess what I'm trying to figure out is this: aside from immutable builds, are there any other fundamental differences when running yarn install locally and in a CI environment?

To reproduce

n/a

Environment

System:
    OS: Linux 5.10 Alpine Linux
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 29.53 GB / 31.36 GB
    Container: Yes
    Shell: 1.34.1 - /bin/ash
  Binaries:
    Node: 17.9.0 - /tmp/xfs-ede05f8e/node
    Yarn: 3.2.0 - /tmp/xfs-ede05f8e/yarn
    npm: 8.5.5 - /usr/local/bin/npm
  Managers:
    pip3: 22.0.4 - /usr/bin/pip3
    RubyGems: 3.3.7 - /usr/local/bin/gem
  Utilities:
    Make: 4.3 - /usr/bin/make
    GCC: 10.3.1 - /usr/bin/gcc
    Git: 2.34.1 - /usr/bin/git
    Clang: 12.0.1 - /usr/bin/clang
  Languages:
    Bash: 5.1.16 - /bin/bash
    Perl: 5.34.0 - /usr/bin/perl
    Python3: 3.9.7 - /usr/bin/python3
    Ruby: 3.1.1 - /usr/local/bin/ruby

(Same environment locally and on CI)

Additional context

No response

[colbymelvin]

We're currently getting around this by using Zero Installs on CI, but I'm still curious about the origin of these lockfile diffs.

[yarnbot]

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟

[seivan]

Sorry we got this issue as well. It's not a big deal and hasn't caused a problem until now when I had the bright idea of adding yarn install --immutable --inline-builds as a git hook. On its own it's not a problem, but it runs tests concurrently and some of the unplugged dependencies seem to change in between even though they shouldn't.

[merceyz]

Why does CI think the lockfile needs to be updated in the first place?

Based on the console output it looks like it's not your lockfile Yarn wants to update, but one of your dependencies. It looks like Yarn is trying to pack one of your dependencies, which I'm assuming is a git repo, using the incorrect version of Yarn causing a problem. Though without a reproduction I can only guess.