[Bug?]: Yarn unable to access binaries in node_modules/.bin

[zachbryant]

Self-service

• [X] I'd be willing to implement a fix

Describe the bug

In a couple of scenarios, yarn cannot access binaries found in node_modules/.bin. One scenario is to use a package such as just-scripts, which should be accessible via yarn just, but is only accessible by the direct installed name yarn just-scripts.

The other scenario is that in package workspaces, even running yarn just-scripts fails despite it being installed and available in node_modules.

To reproduce

Sorry, the link to sherlock doesn't work so my repro won't include it as I'm unfamiliar.

root package.json

{
  "workspaces": [
    "packages/**"
  ],
  "volta": {
    "node": "18.18.2"
  },
  "packageManager": "[email protected]",
  "devDependencies": {
    "just-scripts": "^2.2.1",
    "just-scripts-utils": "^2.0.1"
  },
  "scripts": {
    "build": "just-scripts build"
  }
}

root .yarnrc.yml

nodeLinker: node-modules

root just.config.js

const { task } = require('just-scripts');

task('build', ()=> console.log("Just ran build"))

packages/a/package.json

{
  "packageManager": "[email protected]",
  "scripts": {
    "build": "just-scripts build"
  }
}

1. Run yarn install
2. In the root, run yarn build and observe just-scripts output a message.
3. Now, look into node_modules/.bin and observe entries like just-scripts and just.
4. Run yarn just and observe an error.

For the second scenario,

1. Run yarn install
2. Navigate to packages/a/
3. Run yarn build
4. Observe the error

Environment

System:
    OS: Linux 6.5 Pop!_OS 22.04 LTS
    CPU: (12) x64 AMD Ryzen 5 5600X 6-Core Processor
  Binaries:
    Node: 18.18.2 - /tmp/xfs-963ef41b/node
    Yarn: 4.0.2 - /tmp/xfs-963ef41b/yarn
    npm: 9.8.1 - ~/.volta/tools/image/node/18.18.2/bin/npm
    bun: 0.2.1 - ~/.bun/bin/bun

Additional context

No response

[arcanis]

The just-scripts package doesn't expose a just binary in its metadata: https://yarnpkg.com/package?q=just-scripts&name=just-scripts&file=%2Fpackage.json

[zachbryant]

@arcanis that's true, but the binary does exist in .bin. Even if just-scripts fixes that, any other binary that exists as a result of the installation is still inaccessible via the CLI, such as binaries from its dependencies like flat or semver

[zachbryant]

And let me know if I should make a separate issue for this, but even the bin listed is inaccessible to other workspaces despite its presence in the root

[zachbryant]

Okay so while looking through the code I found -T, --top-level i.e. yarn run -T just-scripts allows packages to access binaries from the root. That unblocks me but I'd like to make the behavior default, so I'm opening a PR shortly and will ping for feedback when it's posted.

And I've just noticed that just is declared by just-task which is a dependency of just-scripts. Seems the two are companion libs, but I've been relying on the yarn v1 behavior described in #5936 to compensate

[arcanis]

This isn't a bug, it's the expected behavior. Yarn enforces that your workspaces only use the dependencies they declare, and that includes scripts.

[zachbryant]

This isn't a bug, it's the expected behavior.

I haven't really followed berry development until late v3 into v4 where the last major blockers for migrating were fixed. So forgive me that it looks like a bug as a v1 user. I'm also fine calling this a feature request instead. But my pain point is it's quite a bit of work to even automate the sorting out of which deps are missing across ~500~ 1000+ package.jsons, given how lenient v1 is.

Yarn enforces that your workspaces only use the dependencies they declare, and that includes scripts.

That said I don't understand the papercut-esque way of enforcing this workspace silo, which I mean to say removing the ergonomic behavior of yarn v1's run pkg vs v4's run -T pkg. The run command can't be shortened even to yarn -T pkg currently.

Yet it looks like an intentional choice that this -T mechanism allows users to escape the workspace and reference root packages. There's also the behavior of allowing unique global scripts which have a colon, but what is the distinction that makes global root scripts disagreeable? I am curious because it seems like a loss of ergonomics for the sake of mostly siloing scripts/deps

[yarnbot]

Hi! 👋

It seems like this issue as been marked as probably resolved, or missing important information blocking its progression. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it.

[yarnbot]

Hi! 👋

It seems like this issue as been marked as probably resolved, or missing important information blocking its progression. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it.