yarn
yarn copied to clipboard
yarn misuses cache on git dependencies
Do you want to request a feature or report a bug? Bug
What is the current behavior? misuses cache to install the git dependency at the wrong commit, seems to happen particularly when specifying git branch
If the current behavior is a bug, please provide the steps to reproduce. sample repo: https://github.com/SEAPUNK/yarn-test-thing
-
yarn add SEAPUNK/yarn-test-thing
- edit package.json's dependency from
SEAPUNK/yarn-test-thing
toSEAPUNK/yarn-test-thing#branch
- edit the yarn.lock file to change the single dependency from what it is to
yarn-test-thing@seapunk/yarn-test-thing#branch:
version "0.0.1-something"
resolved "https://codeload.github.com/seapunk/yarn-test-thing/tar.gz/3ea9fe7984b9005f3cc62b14ee4018013ae0db0f"
i.e. what it will be when you run yarn upgrade
-
yarn install
- inspect node_modules/yarn-test-thing, and see that the files BRANCH_ONE and BRANCH_TWO are not there
What is the expected behavior? installs the right version at commit 3ea9fe7984b9005f3cc62b14ee4018013ae0db0f
Please mention your node.js, yarn and operating system version.
node.js 8.7.0 yarn 1.2.1 macos high sierra
this is particularly annoying when you have committed a commit to the git dependency branch, ran yarn upgrade
, check in the new yarn lockfile to the repo you're working on, only for yarn to not install that "resolved" version in the lockfile when someone else checks the code out and runs yarn install
i'm lazy so i created what i think the tl;dr of the repro steps are, if they don't work, lmk and i'll give you something that should work better
Also just got bitten by this with a GitHub dependency - CI build failed after pushing, even though yarn.lock
was updated with the new commit hash. Worked around the issue for now by making CI pipeline run yarn cache clean
after the other steps in my build.
got bitten by it again; the problem is that yarn is caching the package by its version in package.json, and not by git commit; so it sees in the lockfile that version is 0.0.1-something, looks for the cache for 0.0.1-something of the package, and uses that
the fix for this is to cache these installed-from-git packages by git commit, so in the yarn cache dir it'd look something like git-yarn-test-thing-3ea9fe7984b9005f3cc62b14ee4018013ae0db0f
instead of npm-yarn-test-thing-0.0.1-something
The methodology outline by @SEAPUNK seems sound, installing from a git repository directly would indicate you are interested in having the latest codebase irrespective of versions/releases.
this seemed to be fixed (in 1.7 i think) but since upgrading to 1.9.4 it is back to broken and worse.
in the prior problematic state, i used to be able to bump the version in the target dep to force yarn to update it, but now that's not working and i've been removing the yarn.lock file to get around the issue.
i admittedly forgot about yarn cache clean so can't comment on that atm.
Edit: yarn clean cache
works
Alternative workaround for github dev deps is to define it as URL:
"devDependencies": {
"yarn-test-thing": "https://github.com/seapunk/yarn-test-thing/tarball/3ea9fe7984b9005f3cc62b14ee4018013ae0db0f/yarn-test-thing.tar.gz",
It works around github-specific handlers and just does a plain HTTP download with a checksum (works fine for me so far).
Has this issue been resolved in recent yarn versions? I'm experiencing this problem using yarn 1.6.0 and wondering if I can solve this by upgrading.
I've observed this behavior as well, with @rondale-sc, using both yarn 1.15.2
and yarn 1.13.0
. In our case it appears that the way the dependency is specified mattered. I.e., specifying it as user/repo#SHA
does not update as expected (more details below), but specifying as https://github.com/user/repo.git#SHA
does work as expected.
The situation:
- package.json specifies a dependency using
user/repo#SHA
Github URL form
"devDependencies": {
"my-dep": "user/repo#SHA"
- Someone else makes a commit that updates the dep SHA in package.json and the resolved sha and url in yarn.lock, but the version of the dependency doesn't change.
- I do
git pull
and pick up that commit - I do
yarn install
, and it does not update the package on disk to the one specified by the sha.
I was able to fix this in 3 different ways:
-
rm yarn.lock && yarn install
-
rm -rf node_modules && yarn cache clean && yarn install
- Change the dependency specification to more explicit URL form
https://github.com/user/repo.git#SHA
before making the commit as described above.
Observed with node 8.15
and macOS Mojave.
Updated to point out that we were specifying deps using the short user/repo#SHA
Github URL form, but that it works when using the more explicit URL form https://github.com/user/repo.git#SHA
.
Another update to the previous comment, with some more explicit reproduction steps. I've tested a bit and confirmed that this issue is indeed (at least for me) limited only to usage of the user/repo#SHA
from of the dependency specifier.
To reproduce:
- Assume that there are two separate checkouts of a given repo called
checkout-A
andcheckout-B
. The repo has a package.json with a devDependency pinned to a specific SHA using the github url form, like so:"devDependencies": { "my-dep": "user/repo#SHA-1" }
- The user with
checkout-A
editspackage.json
locally to change the SHA from SHA-1 to SHA-2. The relevant part of their editedpackage.json
now looks like:"devDependencies": { "my-dep": "user/repo#SHA-2" }
- The user in
checkout-A
doesyarn
. This updates the "my-dep" code on disk to SHA-2, as expected, and also modifies theyarn.lock
. The user commits the changedpackage.json
andyarn.lock
and pushes the commit - User in
checkout-B
doesgit pull
and gets the commit. They then doyarn
and the code on disk for "my-dep" does not update to SHA-2 that way it did for user A. - User in
checkout-B
doesyarn cache clean && yarn
and again, the "my-dep" on disk does not update to SHA-2, that way it would be expected to - User in
checkout-B
doesrm -rf node_modules && yarn cache clean && yarn
and now the "my-dep" on disk is updated to SHA-2
I ran through the steps above using yarn 1.15.2
, node 8.15.1
and macOS Mojave.
I also ran through the steps above using explicit URLs in the form https://github.com/user/repo.git#SHA
and the yarn
described in step 4 above did work as expected.
im hitting this issue, bummer as i was trying to move our whole CI process to use yarn.
the error for me is with tags, not specific commit hash:
error Command failed.
Exit code: 1
Command: git
Arguments: fetch --tags
Directory: c:\yarn-cache\v4\.tmp\4f6877135181ea234579e349f25b0ce3
Output:
From https://gitlab.com/xxx/dashboard
! [rejected] v1.0.x -> v1.0.x (would clobber existing tag)
! [rejected] v1.x.x -> v1.x.x (would clobber existing tag)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
I was bitten by this as well. @bantic's observation seems correct, it doesn't occur when the dependency is specified as https://github.com/user/repo.git#SHA
instead of user/repo#SHA
(same for specifying a branch instead of a SHA in package.json).
Just ran into this issue. Thanks to @bantic I was able to work around it using this method:
- Change the dependency specification to more explicit URL form
https://github.com/user/repo.git#SHA
before making the commit as described above.
Note: I was also able to work around this issue by running yarn clean cache
before running yarn install
each time, but the above solution does not require additional steps in the build process.
Any news on this issue ? I feel we all lost quite a bit of time with this since it was open several years ago.
It would be so great if it could be fixed !
Not holding much hope for a resolution at this point, but we're getting bitten by this too. In our case we're using a git+ssh://git@...
-style URL, pointed at gitlab. Based on what I'm seeing elsewhere in the thread maybe we'll try an https URL to see if that helps