berry
berry copied to clipboard
[Bug?]: Checksum mismatch for github package - using yarn v1 to pack
Self-service
- [ ] I'd be willing to implement a fix
Describe the bug
When installing a package from github "cypress-log-to-output": "Narretz/cypress-log-to-output#feat-no-output"
, I'm getting a The remote archive doesn't match the expected checksum
error in CI (CircleCI - docker image cimg/node:16.13.2
). I'm using the node modules linker, not PnP.
I've compared the contents of the package it's creating in /tmp
on CI with the contents of the package in my local (macOS) yarn cache. The only file with checksum differences is package.json
. In my local version, the package.json
has the trailing newline (which matches the git repo). The version on CircleCI is trimming the trailing newline as part of the yarn pack --install-if-required
step (the git clone still has the newline before and after the yarn pack
, but the version inside the tarball seems to have it trimmed).
From looking at the source, I can see that yarn berry will always be trimming this trailing newline and I can reproduce it locally.
If I switch from the github protocol to the git one, I get the same result, but can see the package being regenerated:
➤ YN0000: ┌ Resolution step
➤ YN0013: │ cypress-log-to-output@[email protected]:Narretz/cypress-log-to-output.git#commit=58931ea7110dfd7779814f699aa12006eea93585 can't be found in the cache and will be fetched from the remote repository
➤ YN0000: │ /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2 STDOUT Packing cypress-log-to-output@[email protected]:Narretz/cypress-log-to-output.git#commit=58931ea7110dfd7779814f699aa12006eea93585 from sources
➤ YN0000: │ /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2 STDOUT No package manager configuration detected; defaulting to Yarn
➤ YN0000: │ /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2 STDOUT
➤ YN0000: │ /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2 STDOUT yarn pack v1.22.15
➤ YN0000: │ /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2 STDOUT success Wrote tarball to "/private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2/package.tgz".
➤ YN0000: │ /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-fc04a1e2 STDOUT Done in 0.06s.
Note specifically yarn pack v1.22.15
I'm definitely running yarn berry, but for the yarn pack
inside it, it's using the globally installed yarn 1. I'm guessing yarn 1 isn't trimming the trailing newline, so then we get a mismatch.
In the source, in yarn 2 mode it seems to be doing a lot to try to get this environment set up right, but for some reason (nvm?) it isn't working for me locally. It does seem to the right thing in my CI environment. I used corepack enable
to set up berry - maybe that isn't ending up being carried across when spawning the separate yarn process when using nvm?
This seems like some missing piece of local environment config, but I've followed the getting started docs and can't see anything else missing.
To reproduce
Can't reproduce with sherlock - depends on the yarn 1 install
- Create new package with
yarn init -2
- Add a github dependency which doesn't define a package manager
{
"name": "repro",
"packageManager": "[email protected]",
"dependencies": {
"cypress-log-to-output": "Narretz/cypress-log-to-output#feat-no-output"
}
}
- Set
enableInlineBuilds: true
so that you can see the pack output - Run
yarn install
- Note that it uses yarn 1.x to do
yarn pack
on the github dependency
Environment
System:
OS: macOS 10.15.7
CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Binaries:
Node: 16.13.0 - /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-a816c7ae/node
Yarn: 3.1.1 - /private/var/folders/r5/lxs794h569vg8ntxmrn8jrvm0000gp/T/xfs-a816c7ae/yarn
npm: 8.1.0 - ~/.nvm/versions/node/v16.13.0/bin/npm
npmPackages:
jest: ^27.2.5 => 27.4.5
Additional context
No response
Hi! 👋
This issue looks stale, and doesn't feature the reproducible
label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).
Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃
If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded
label. Thanks for helping us triaging our repository! 🌟
I had the same issue running [email protected] (current stable version) installed with corepack.
Installing the package "canvas": "dwickern/canvas-noop"
from yarn 1.22.19:
canvas-https-d4dab75653-31cdc4ba4d.zip
The same package installed with yarn 3.2.2 via corepack without the yarn v1 bootstrapper: canvas-https-d4dab75653-6b6d48e4ad.zip
Workaround is to corepack disable
and install yarn v1.
I have encountered the same problem while upgrading from Yarn v1 to v3.2.1 recently.
In package.json
(only relevant parts):
{
"packageManager": "[email protected]",
"dependencies": {
"timecop": "git+https://github.com/jamesarosen/Timecop.js.git#d421df9b00386266bda93eb9c2ea15fedc888761"
}
}
When installing on my machine (Mac OS) with corepack enabled:
$ rm -rf node_modules && yarn cache clean --all && yarn | grep '/private/var/folders/fk'
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f STDOUT Packing timecop@https://github.com/jamesarosen/Timecop.js.git#commit=d421df9b00386266bda93eb9c2ea15fedc888761 from sources
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f STDOUT No package manager configuration detected; defaulting to Yarn
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f STDOUT
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f STDOUT yarn pack v1.22.15
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f STDOUT success Wrote tarball to "/private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f/package.tgz".
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-7e3c703f STDOUT Done in 0.18s.
With corepack disabled:
$ rm -rf node_modules && yarn cache clean --all && yarn | grep '/private/var/folders/fk'
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT Packing timecop@https://github.com/jamesarosen/Timecop.js.git#commit=d421df9b00386266bda93eb9c2ea15fedc888761 from sources
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT No package manager configuration detected; defaulting to Yarn
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: .jshintrc
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: Contributing.md
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: Gemfile
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: Gemfile.lock
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: LICENSE
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: README.md
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: Rakefile
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: lib/BuildTemplate.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: lib/MockDate.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: lib/TimeStackItem.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: lib/Timecop.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: lib/timecop/compilation_task.rb
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: lib/timecop/jshint_task.rb
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: package.json
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/MockDateSpec.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/TimeStackItemSpec.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/TimecopSpec.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/helpers/SpecHelper.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/support/jasmine-html.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/support/jasmine.css
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/support/jasmine.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/support/jasmine.yml
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/support/jquery-1.4.4.min.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: spec/javascripts/support/json2.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: timecop.js
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: Package archive generated in /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064/package.tgz
➤ YN0000: │ /private/var/folders/fk/c9z1tyhj02nfdn9dz_71vjk80000gq/T/xfs-88fd0064 STDOUT ➤ YN0000: Done in 0s 112ms
The resulting ZIP files are almost exactly identical, save for one difference: an extra newline in the package.json
. Note that, in both cases, the version of Node and Yarn CLI is exactly the same. The only difference is that when corepack is enabled/disabled, different versions of Yarn are available (I assume), which means our Git dependency is built with a slightly different version of Yarn, which means the newline is removed / added, which means the checksum changes, which means the install fails. In summary: depending on the Yarn versions available in your environment, the checksums of Git deps may be different.
I ran into this in practice in 2 instances:
- I installed a package on my machine (corepack enabled). Install fails on the CI (
node:16.15.1-alpine
, corepack disabled by default) - I installed a package on my machine (corepack enabled). A colleague pulls my changes to their machine (corepack disabled)
For the time being, we'll avoid using corepack at all so all environments are the same.