turbo
turbo copied to clipboard
[turborepo] lockfile changes causes all packages to be "changed" when using `--filter=[HEAD~1]`
What version of Turborepo are you using?
1.6.3
What package manager are you using / does the bug impact?
Yarn v1
What operating system are you using?
Mac
Describe the Bug
When any change is added to yarn.lock
(even whitespace), running npx turbo run <command> --filter=[HEAD~1]
causes the command to be run in every package.
Expected Behavior
I know this is probably a hard one to fix.
But I would prefer if turbo
would parse the yarn.lock
file and determine if the changed package affects a workspace before executing the command.
To Reproduce
- Create a new turbo repo
- Make a change to the
yarn.lock
(even a whitespace is fine) - Commit this change
- Run
npx turbo run build --filter=[HEAD~1]
Reproduction Repo
No response
I'm going to have a look at fixing this - beacuse IMO this is a critical flaw in turborepo. It's very strange especially because the docs seem to acknowledge this as an issue with monorepos .
So you'd think this would be a core pillar of the design of turborepo.
Right now when I add any package to my repo - I have to re-deploy 10 apps which can take my GHA build up to 45 minutes (not to mention the potential outage of re-deploying 9 unchanged services)
@dan-cooke did you find any solution or a workaround for this issue ? I have the same problem.
@Marwennnne there is no solution or workaround - I have not got round to fixing it in turbo repo yet. I’m just taking the hit in CI runtime
I appreciate this isn't a helpful addition but we're also running into this problem. We're having to re-build and re-deploy all of our apps whenever a dependency is added or changed in any of them.
Issue appears also on yarn v3.2.4
Very interested in seeing a fix for this as well (we are using pnpm-lock.yaml
)
Facing the same issue with pnpm-lock.yaml
.
any update on this ?
Would the team accept a PR for this? I have no idea if it's something I could manage, but it might be interesting to take a stab at it.
Update: I attempted to investigate this, but I couldn't actually reproduce the issue specifically! I wonder if it was fixed at some point. We're on v1.10.16.
Still hapenning here, turbo 1.10.16
and pnpm 8.10.2
Just to let you know this is also happening in the last version 1.11.2
(with pnpm 8.7.5
)
Works in 1.13.1-canary.3
(with pnpm 7.33.6
)
Wonder if it's because using a shared workspace lock file.
Noticed when doing a package update (or adding some package) that a lot of other packages use I can see pnpm
resolve and dedupe in other dependencies. So if I add something that affects the downstream of packages it'll mark as affect. So it is technically correct to mark as affected (just hard to realize sometimes).
If I add a package to a monorepo that I know will not affect any other libs (I used left-pad
), filter works as expected and only marks the package I added it in as changed (this still modifies root pnpm-lock.yaml).
I didn't validate if I can find some package that doesn't affect all but more than 1.
Hey, folks! This feature has been a part of Turborepo since 1.7. Sorry for only coming across this issue just now!
For the reports that are on this issue that are using 1.7 or later, we're suspecting that dependencies are being updated deep in the tree, resulting in cache misses, as @EWhite613 has astutely noted. I just did a few similar tests as @EWhite613 and found the expected behavior.
Much of the caching logic around dependencies is drawn off of this logic so we're pretty confident that we have this working how it should. If you're still convinced you're seeing unexpected cache misses, please open a fresh issue with a reproduction so we can take a look.
Yes, in our investigations, we've more or less isolated the problem to pnpm unexpectedly updating sub-sub-sub-sub-etc dependencies shared by many projects, rather than an issue in Turborepo's logic.
Yes, in our investigations, we've more or less isolated the problem to pnpm unexpectedly updating sub-sub-sub-sub-etc dependencies shared by many projects, rather than an issue in Turborepo's logic.
any way of preventing this behavior from pnpm ? It's a pain to run the whole CI because I've update one package somewhere
Also suffering from this issue, with pnpm v9.0.6 and also with earlier versions on pnpm, turbo v1.13.3 Tried to upgrade to v2.0.4 but still had the same issue Most of the PRs that changes the pnpm-lock.yaml triggers the entire mono-repo tasks