Workspaces don't run lifecycle scripts when linking local packages

[jquense]

What is the current behavior?

When you do a yarn install in a workspaces enabled project, yarn doesn't run the various lifecycle scripts for workspace packages, e.g. prepare prepublish preinstall etc

If the current behavior is a bug, please provide the steps to reproduce.

Here is a commit adding a failing test case: https://github.com/jquense/yarn/commit/61eaf4239f707f772af5713ebc98963589a6dad4

What is the expected behavior?

Lifecycle scripts are run for the local packages so they have an opportunity to build steps. This is what lerna does.

Please mention your node.js, yarn and operating system version.

yarn 27.5 node 8 Mac Sierra

cc @bestander

[bestander]

PR is welcome BTW :)

[jquense]

I took a stab at it, but could figure out the code base unfortunately. All I can see is that scripts is empty for the linked packages when it gets to the getInstallCommands in the pacakge-install-scripts :/

[bestander]

Thanks for giving it a try, @jquense. The entry point is probably here https://github.com/yarnpkg/yarn/blob/master/src/cli/commands/install.js#L477 - flattenedTopLevelPatterns only contains the top level dependencies of root package.json + a virtual dependency that refers all workspaces. I think going deeper PackageInstallScripts should handle that https://github.com/yarnpkg/yarn/blob/master/src/package-install-scripts.js#L248.

If you want you can start with submitting a PR with a failing test and we can sort it out in a follow up.

[jquense]

The failing test case linked in the issue is the best I can do sorry! Unfortunately I just don't have the time to spend hours learning the yarn codebase to try and fix this.

[BurtHarris]

Hi. I'm quite interested in yarn and workspaces in particular. I'll be giving a look at this as a possible first contribution.

[bestander]

Awesome! If you have any questions ask away

On Mon, Jul 31, 2017 at 7:28 PM Burt Harris [email protected] wrote:

Hi. I'm quite interested in yarn and workspaces in particular. I'll be giving a look at this as a possible first contribution.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yarnpkg/yarn/issues/3911#issuecomment-319250162, or mute the thread https://github.com/notifications/unsubscribe-auth/ACBdWBmqV2fFlBlOvOlXB77Ml0213Xw6ks5sTo1UgaJpZM4OUyrV .

[BurtHarris]

@bestander I hit a snag right away, Issue #4059. Its almost certainly related to the fact I'm running Windows any hints on how to make that work?

[bestander]

As a workaround you should be able to run tests like this node ./node_modules/jest/bin/jest.js

[jquense]

Any word from anyone? I might have some time to jump, and I really would love to get this fixed :) I have to think that for most folks the lack of lifecycle scripts running is a dealbreaker for this feature?

[bestander]

The core team is focused on fixing the most high priority issues right now, cases where Yarn may crash or install wrong versions of dependencies.

This bug will be resolved eventually but considering that this is a community project the quickest way would be to spend time learning the codebase and sending a fix.

[jquense]

Definately :) I'm not trying to be pushy or entitled just wanted to get a sense of where the issue was and if there was any existing work

[ryanhiebert]

FWIW, it's not the blocker for my adoption of workspaces. I haven't figured out how to reliable add and remove dependencies (not unlikely to be user error). Handling this was annoying, but I just needed to add calls to the packages' prepare scripts to the parent package.json prepare script, which was a good enough workaround for me.

[nkbt]

Some more findings (yarn workspaces + lerna) that might be helpful

With workspaces lerna bootstrap is unnecessary and does nothing, so we just run prepublish after initial install

yarn install --pure-lockfile
yarn lerna run prepublish

Lerna is smart enough to run prepublish in correct order so libs are not getting build before their deps.

The only thing is you should not enforce parallel lerna build like yarn lerna run prepublish -- --parallel. In this case you may get build errors when order matters.

[jquense]

I've also found the prepublish lerna approah works well. However the one problem with the lerna run approach is packages that define binaries that other packages use in their prepublish scripts. Lerna runs them in the correct order, but because the the prepublish creates the bin file after the install process is run none of the other local packages have that bin linked in thier .bin folders and so break.

[Hypnosphi]

@jquense my workaround was to create additional entries that just require the actual binaries. Those entries are present at the moment of installation, so yarn has something to link. See https://github.com/storybooks/storybook/pull/1810/commits/4d10a559978c5e1d1391b022e7110576c5c71a0d#diff-6c1388595b10c4ae5635b68cf08edc98

[darkobits]

This behavior is currently making it impossible to migrate from Lerna's bootstrap to Yarn Workspaces.

It is imperative that the prepare/prepublish scripts are run (and in the correct order in each package, based on the dependency graph) to ensure that linked dependencies are built and available to dependents.

[bestander]

Sounds reasonable, send a PR

[martijnthe]

As a work-around, what seems to work for us is to add the lifecycle scripts to the root package.json and use lerna to "forward" them to the packages in the workspace, so for example:

repo_root/package.json:

"scripts": {
  "prepare": "lerna run --stream --sort prepare"
  ... etc ...
}

If yarn install is run in the root, prepare will get run for all the packages in the workspace.

[martijnthe]

Incidentally, I'm now running into: https://github.com/yarnpkg/yarn/issues/4973 Is that the same issue perhaps?

The work-around I had posted above ^^^^ doesn't work for packages that aren't part of the workspace.

[stipsan]

Is anybody working on this? If not I'll give it a shot 😄

[bestander]

Go for it

[ccapndave]

For my (very simple) use-case I need a top-level prepare to run prepare in a specific workspace, so in the root package.json I added:

{
  "scripts": {
    "prepare": "yarn workspace my-workspace run prepare"
  }
}

Not very pretty, but gets the job done for the moment.

https://www.npmjs.com/package/wsrun might also be useful.

[FezVrasta]

I'm sorry to bump this but is anyone actually working on this? Workspaces are kinda half baked until this feature is implemented.

[stipsan]

I haven't had time to start on it yet 👨‍💻

[stipsan]

I hope I don't deter anyone else from starting, so unless I update this thread and say I'm working on it please assume that I'm not working on it 😅

[NathanielHill]

I'm using @ccapndave technique to trigger prepare in a shared react component library that needs transpilation. Would be nice if #6869 could be merged, even if it's only for prepare and prepublish as there are ordering issues with the others.

[thejohnfreeman]

I'm using @martijnthe's solution above and when I yarn, the top-level prepare script runs twice. When I yarn prepare, it only runs once, as expected. Does this happen to anyone else?

  "scripts": {
    "prepare": "lerna run prepare"
  }

[FezVrasta]

For everybody in need of this feature. Take a look at lerna-alias. It's a good way to avoid to need to build the workspace packages during developement

[mike-marcacci]

Came here looking for a different problem, but wanted to note that I don't terribly mind the current state of things, as it allows me to control which packages' scripts get run. Imagine a really, really big monorepo: you wouldn't necessarily want to run every prepare script just to work on one!

If you want to simulate the proposed behavior, you can quite easily do so in the corresponding lifecycle script in the top-level package.json:

{
  // ...
  "scripts": {

    // This runs the `prepare` script in each package.
    "prepare": "for P in packages/*/; do echo \"\n\n---------- $P\" && (cd $P && yarn prepare); done"

    // This runs the `test` script in each package, but exits on the first failure
    "test": "for P in packages/*/; do echo \"\n\n---------- $P\" && if ! (cd $P && yarn test); then exit 1; fi; done"
  }
}

I think any solution that changes the current behavior would need a way to opt out.

[pladaria]

@mike-marcacci if you want to avoid building already built packages, then skip build process if already built

yarn should execute all lifecycle scripts because you don't know what dependencies your package has. The whole repo must be built.