Support monorepos

[mjsisley]

Thanks for the great lib!

Currently, doesn't detect changes in nested package.json and yarn.lock. What do you think about adding support for monorepos?

[frontsideair]

I'd love it, but I don't have any experience with monorepos. I'd love a PR, but if you can outline a plan that would be good too, and I might get to do it when I have time.

[mjsisley]

This tool supports monorepos https://github.com/renovateapp/renovate and could be a reference.

Basically, a monorepo has subdirectories that also contain package.json and lockfiles. Traversing the sub-directories (ignoring node_modules) and performing the code already performed in this repo in any subdirectory that has a package.json and a lockfile should do the trick.

[frontsideair]

I had some time today and took a look at what can be done. I'd like to keep this tool simple so I tried to see what are the common traits of monorepos. Looks like most of them use lerna as the bootstrapping tool, and it's easy to detect with lerna.json present on the project root. Running lerna bootstrap seems to install all the dependencies.

I can implement today or in a few days or you can do it if you'd like it. It will be good to support this use case too. Thanks for the suggestion.

[frontsideair]

It turned out to be trickier than I imagined. lerna bootstrap installs dependencies of subpackages but it doesn't install its own dependencies, so running yarn install is necessary.

Also the current strategy for detecting a diff works only for a lockfile adjacent to a package.json so deeper changes are not detected. I'll think more about this.

The current code is in lerna branch, feel free to check it out.

[frontsideair]

I found how renovate handles monorepos: https://github.com/renovateapp/renovate/blob/fc4d5fc08d966c54e319c42e9acb9da89c0dd0df/lib/manager/npm/monorepos.js Looks like it currently supports lerna and yarn workspaces.

I think traversing all subdirectories can be expensive, this command must be very quick or it makes switching branches a pain. (A coworker experienced this internally, before we started checking for diff in lockfile.) It makes sense to get a list of subpackages from the lerna.json config and use lerna to bootstrap them. I also investigated different monorepo tools and came up with a list:

• Yarn workspaces: I think this works out-of-the-box but I need to test
• Lerna: Primary target
• Bolt: I think it's not officially released but it's probably gonna be big
• Builder: Formidable is a big OSS contributor but I don't know how widely it's used
• Rush: This is by Microsoft, same concerns
• northbrook: No longer developed I guess
• Bazel, Buck, etc: These are non-js monorepo tools, so they have less priority (Angular uses Bazel though)

[cinderblock]

What if there was an optional argument or yarnhook.json config file that specified a list of directories or globs to scan for changes? Possibly recursively loading yarnhook.json files in found directories?

Sidebar: I would not even try to target specific monorepo tooling. That's a can of worms that you'd never be able to close

[Skylion007]

Yeah, just being able to detect an package.json in a subfolder would be great!