pnpm patches incorrectly excluded on turbo prune

[kalvenschraut]

Verify canary release

• [X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/RTVision/turborepo-prune-patch-issue

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Which canary version will you have in your reproduction?

2.1.2-canary.0

Describe the Bug

patches are being remove from the pruned workspace when app depends on the package.

Expected Behavior

all patches for dependent libraries be included in the prune of an app.

To Reproduce

clone the repo and run pnpm turbo prune --scope=web and then open up out/pnpm-lock.yaml and see the next patch is no longer present even though it is the base pnpm-lock.yaml file and web depends on next

Additional context

pnpm recently added a feature that allows trying to apply the same patch to all versions of a package which was added in pnpm 9.7. Because of that the patch doesn't have a version in the file name or in the lockfile which I suspect is causing not to be added during the prune. This is the default way pnpm generates patch files now so any new patches done are no longer being kept around.

[liam-germain]

hi is this bug related to why i can't run even the 'with-docker' example with it failing on:

web:build: cache miss, executing e9bd4163eb4a2003 0.347 x internal errors encountered: unable to find package manager binary: cannot 0.347 | find binary path

Everything works great when using yarn but fails when using pnpm

[ideatrails]

I too just sunk too much time trying to integrate the with-docker starter example into an existing repo running on pnpm. I finally tried the raw examples and discovered both the npm and yarn versions work, but as liam-germain suggests, the with-docker example does not work with pnpm.

[nathan2slime]

I've been struggling with this issue all day. I can't get my app to run in Docker when using the standalone output. It seems like the dependency directories inside the Docker image are created as symbolic links, which is causing issues during runtime

[Schroedi]

This is my workaround:

# copy the patchedDependencies from the root package.json to the pruned package.json
RUN jq --slurpfile src <(jq '.pnpm.patchedDependencies' package.json) '.pnpm.patchedDependencies = $src[0]' /app/out/full/package.json > package.json.patched && mv package.json.patched /app/out/full/package.json

Make sure jq is installed and maybe you need to adjust some paths.

[nathan2slime]

I solved it, it was just a pnpm-lock.yaml that was in my apps/web dir

[icelattefull]

I'm also experiencing this issue with turbo 2.3.1. In my case, in addition to the solution provided here, I also had to:

• patch the pnpm lock file as I'm using pnpm install --frozen-lockfile
• copy the patches folder, which is also ignored in the prune command

COPY patches /application/out/json/patches
COPY patches /application/out/full/patches

RUN jq --slurpfile src <(jq '.pnpm.patchedDependencies' package.json) '.pnpm.patchedDependencies = $src[0]' /application/out/json/package.json > package.json.patched && mv package.json.patched /application/out/json/package.json
RUN yq eval-all 'select(fileIndex==0).patchedDependencies = select(fileIndex==1).patchedDependencies' out/json/pnpm-lock.yaml pnpm-lock.yaml > out/json/pnpm-lock.yaml

[puckey]

I'm also experiencing this issue with turbo 2.3.1. In my case, in addition to the solution provided here, I also had to:

• patch the pnpm lock file as I'm using pnpm install --frozen-lockfile
• copy the patches folder, which is also ignored in the prune command

COPY patches /application/out/json/patches COPY patches /application/out/full/patches

RUN jq --slurpfile src <(jq '.pnpm.patchedDependencies' package.json) '.pnpm.patchedDependencies = $src[0]' /application/out/json/package.json > package.json.patched && mv package.json.patched /application/out/json/package.json RUN yq eval-all 'select(fileIndex==0).patchedDependencies = select(fileIndex==1).patchedDependencies' out/json/pnpm-lock.yaml pnpm-lock.yaml > out/json/pnpm-lock.yaml

I had to adjust the yq command in order for it not to output both documents combined:

COPY patches /application/out/json/patches
COPY patches /application/out/full/patches

RUN jq --slurpfile src <(jq '.pnpm.patchedDependencies' package.json) '.pnpm.patchedDependencies = $src[0]' /application/out/json/package.json > package.json.patched && mv package.json.patched /application/out/json/package.json
RUN eval-all 'select(fileIndex==0) as $f1 | select(fileIndex==1) as $f2 | $f1.patchedDependencies = $f2.patchedDependencies | $f1' out/json/pnpm-lock.yaml pnpm-lock.yaml > out/json/pnpm-lock.yaml

[gajus]

Just flagging to the core team that this is a major issue that currently seems to be ignored.

@anthonyshew @chris-olszewski

[gajus]

here is a complete patch that worked for us:

# patch start -- https://github.com/vercel/turborepo/issues/9120#issuecomment-2580436623
RUN apt-get update && apt-get install -y jq
RUN wget https://github.com/mikefarah/yq/releases/download/v4.40.5/yq_linux_amd64 -O /usr/bin/yq && chmod +x /usr/bin/yq

COPY patches /srv/out/json/patches
COPY patches /srv/out/full/patches

RUN bash -c 'jq --slurpfile src <(jq ".pnpm.patchedDependencies" package.json) ".pnpm.patchedDependencies = \$src[0]" /srv/out/json/package.json > /srv/out/json/package.json.patched && mv /srv/out/json/package.json.patched /srv/out/json/package.json'

RUN yq eval-all 'select(fileIndex==0).patchedDependencies = select(fileIndex==1).patchedDependencies' /srv/out/json/pnpm-lock.yaml /srv/pnpm-lock.yaml > /srv/out/json/pnpm-lock.yaml
# patch end

[gajus]

@chris-olszewski I saw related PR made by Devin. Is this fixed/was it released?

[kykungz]

@gajus Updated to Turbo v2.5.4 — pnpm’s patched packages are finally working like a charm!

[pepijn-vanvlaanderen]

On v2.5.4 still having issues. I do see patches in the out dir and the patchedDependencies are in the lock file. Still run into errors like:

#17 0.499 Lockfile is up to date, resolution step is skipped
#17 0.532  ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY  Broken lockfile: no entry for '@strapi/[email protected](patch_hash=63581ce8c3fe5e057e0d5787a30f89c0e6f403cb5d05d952ba59962f21bddd67)(@swc/[email protected])(@types/[email protected])(@types/[email protected])(@types/[email protected](@types/[email protected]))(@types/[email protected])([email protected])([email protected]([email protected]))([email protected]([email protected]([email protected]))([email protected]))([email protected])([email protected])([email protected]([email protected]([email protected]))([email protected]))([email protected])([email protected])' in pnpm-lock.yaml

[nickythorne]

Using [email protected] the docker prune web --docker is successfully retaining the patchedDependencies in the out/json/pnpm-lock.yaml file. However the output package.json has them removed. As a result the pnpm install gives a error due to mismatch.

out/pnpm-lock.yaml
patchedDependencies:
  next-auth:
    path: patches/next-auth.patch
    hash: 99c9ceea7b9d0d4b7d0f57f65d4b3c39e96329dc4432f40dc41090c552e2146f

out/package.json
  "pnpm": {
    "patchedDependencies": {},

[anthonyshew]

Hey, folks, just realized this issue is still open. We can close it with https://github.com/vercel/turborepo/pull/10430 released.

Also, just to address much of the conversation on this issue:

• The reproduction in the issue contains a broken lockfile when I try it?
• @liam-germain @ideatrails: The difficulty with the with-docker example is that it is not prepared for pnpm out of the box. You need to install pnpm in the Dockerfiles for pnpm to be in the container. I realize this is less than perfect DX, and we can do something to fix, but it is indeed unrelated to the issue.
• @icelattefull @gajus: You should be able to use turborepo without those scripts with https://github.com/vercel/turborepo/pull/10430 being a part of Turborepo today. If that is not the case, please open a new issue with a reproduction so that we can fix.
• @pepijn-vanvlaanderen @nickythorne: What you're seeing is likely a separate issue from what OP was trying to report (though it appears to rhyme). Please open an issue with a reproduction so we can fix.

Thanks, all!