[turborepo] `turbo prune` does not copy over `file:` dependencies in root `package.json`

[dhermes]

What version of Turborepo are you using?

1.8.3

What package manager are you using / does the bug impact?

Yarn v1

What operating system are you using?

Linux / macOS

Describe the Bug

Running yarn install --ignore-scripts --prefer-offline --frozen-lockfile fails unexpectedly:

After running yarn install --ignore-scripts --prefer-offline (i.e. without --frozen-lockfile) and tracking the change to the yarn.lock, we see that the pruned yarn.lock is missing file: deps:

--- out/yarn.lock	2023-03-07 10:48:35
+++ out/yarn.lock.new.txt	2023-03-07 10:48:30
@@ -1982,6 +1982,17 @@
   resolved "https://registry.yarnpkg.com/@graphql-typed-document-node/core/-/core-3.1.2.tgz#6fc464307cbe3c8ca5064549b806360d84457b04"
   integrity sha512-9anpBMM9mEgZN4wr2v8wHJI2/u5TnnggewRN6OlvXTTnuVyoY19X6rOv9XTqKRw6dcGKwZsBi8n0kDE2I5i4VA==
 
+"@hardfin/eslint-config@file:./project/packages/eslint-config":
+  version "0.0.0"
+  dependencies:
+    "@typescript-eslint/eslint-plugin" "5.54.0"
+    "@typescript-eslint/parser" "5.54.0"
+    eslint-config-next "13.2.2"
+    eslint-config-prettier "8.6.0"
+    eslint-plugin-no-relative-import-paths "1.5.2"
+    eslint-plugin-no-type-assertion "1.3.0"
+    eslint-plugin-unused-imports "2.0.0"
+
 "@humanwhocodes/config-array@^0.11.8":
   version "0.11.8"
   resolved "https://registry.yarnpkg.com/@humanwhocodes/config-array/-/config-array-0.11.8.tgz#03595ac2075a4dc0f191cc2131de14fbd7d410b9"
@@ -9452,7 +9463,7 @@
   resolved "https://registry.yarnpkg.com/typescript/-/typescript-4.9.3.tgz#3aea307c1746b8c384435d8ac36b8a2e580d85db"
   integrity sha512-CIfGzTelbKNEnLpLdGFgdyKhG23CKdKgQPOBc+OUNrkJ2vr+KSzsSV5kq5iWhEQbok+quxgGzrAtGWCyU7tHnA==
 
[email protected], typescript@^4.3.5:
+typescript@^4.3.5:
   version "4.9.5"
   resolved "https://registry.yarnpkg.com/typescript/-/typescript-4.9.5.tgz#095979f9bcc0d09da324d58d03ce8f8374cbe65a"
   integrity sha512-1FXk9E2Hm+QzZQ7z+McJiHL4NW1F2EzMu9Nq9i3zAaGqibafqYwCVU6WyWAuyQRRzOlxou8xZSyXLEN8oKj24g==

Expected Behavior

turbo prune should pick up all dependencies both in the --scope of the build as well as the root package.json. Following the recommendation, we are pruning, copying the yarn.lock into the repository root and then running yarn install from the root. However this means any dependencies in the root package.json need to be present during yarn install.

The lockfile being pruned does contain this file: dependency and others

$ grep -n 'file:' package.json
18:    "@hardfin/eslint-config": "file:./project/packages/eslint-config",
$ grep -n 'file:' yarn.lock 
2080:"@hardfin/eslint-config@file:./project/packages/eslint-config":
2091:"@hardfin/jest-presets@file:project/packages/jest-presets":
2094:"@hardfin/tsconfig@file:project/packages/tsconfig":
2097:"@hardfin/ui@file:project/packages/ui":

but the pruned lockfile does not:

$ cat out/yarn.lock | grep 'file:'

and then after letting yarn add the dependency, the pruned lockfile has the only file: dependency used for the given project:

$ cat out/yarn.lock.new.txt | grep 'file:'
"@hardfin/eslint-config@file:./project/packages/eslint-config":

To Reproduce

Use a file: dependency in a root package.json managed by turbo, then run

turbo prune --scope=@foo/bar --docker

mkdir tmp-install/
cp out/yarn.lock tmp-install/
cp -r out/json/* tmp-install/
cd tmp-install/
yarn install --ignore-scripts --prefer-offline --frozen-lockfile # Expected to fail
yarn install --ignore-scripts --prefer-offline

cd ..
mkdir tmp-build/
cp -r out/full/* tmp-build/
mv tmp-install/node_modules/ tmp-build/
rm -fr tmp-build/
cd tmp-build/
yarn turbo run build --filter=@foo/bar

Reproduction Repo

No response

[arlyon]

Hey! Thanks for providing a repro. After going through the steps I also hit this issue. We are going to address this (unless you are wanting to open a PR), and I will get a proper timeline soon :)

Thanks for using turborepo!

[dhermes]

Awesome thanks!

[dhermes]

@arlyon Any sense of a timeline?

[xpluscal]

@arlyon I second this - any potential fix?

[arlyon]

Hey folks, we finished up our rust porting work a few weeks back and are re-evaluating old bug / feature requests. Apologies for the stall, but we hit a point where applying fixes while actively re-writing it didn't make sense.

That said, we're back! Can someone help me understand why you need local file deps rather than using the workspace mechanism of your package manager?

[danielpza]

Can someone help me understand why you need local file deps rather than using the workspace mechanism of your package manager?

Some enterprise libraries provides you with a zip file of their package instead of putting it in a registry