[feature] an easy way to add NPM packages, dependencies and/or devDependencies as inputs

[niieani]

Is your feature request related to a problem? Please describe.

It's really hard to define inputs in a monorepo when you depend on other modules.

Here's an example:

• A monorepo package webpack-config which includes my webpack.config.js
• A monorepo package app which uses the webpack.config.js from the webpack-config module in the monorepo
• my app has a task that runs webpack with the config from webpack-config.

Doing any of those things example tasks should invalidate the cache:

• editing the webpack.config.js in webpack-config project within the monorepo
• upgrading webpack devDependency in the project root
• upgrading webpack dependency in the webpack-config
• upgrading swc-loader dependency in the webpack-config, as it is used by webpack.config.js

Describe the solution you'd like

It would be great if we could define that a given task depends on the nested dependency tree of a given package, or specific files from any NPM dependency, using the node_modules resolution algorithm, e.g.:

inputs:
  - @dependenciesOf(webpack-config)
  - webpack-config/webpack.config.js

I'm guessing the first one would be simpler to implement, since it only needs the dependency tree, and the latter one would also need a node resolution algorithm.

One thing to keep in mind is that if a given package exists as a project inside of the monorepo, any change to any file of that project should also invalidate the cache locally.

Describe alternatives you've considered

I guess we could add all of node_modules as inputs, but that's a little nasty, esp. when you need to add in the root devDependencies too, or dependencies of another package:

inputs:
  - **/*
  - ../other-package/**/*

Gets pretty tedious if you have to maintain the list manually.

And doesn't even work, because moon responds with:

Error: config::parse::failed

  × Failed to parse packages/platform/cli/moon.yml.
  ╰─▶   × tasks.build-library-bundle-cli.inputs[2]: .: parent relative paths are not supported

Additional context

[milesj]

To start, package.json dependencies are already included in the hashing process of a task, but only for dependencies of the project the task resides in. You can verify this by inspecting the JSON hash files.

Invalidating based on package.json changes in external projects you depend on is quite difficult, and maybe impossible? with the current architecture. However, there is a "trick" to get around this. Simply have the task depend on a fake task in the external projects.

For example, in webpack-config, have a fake/noop task.

tasks:
  invalidate:
    command: noop

And in the consuming projects, depend on that task.

tasks:
  build:
    deps: ['webpack-config:invalidate']

[niieani]

package.json dependencies are already included in the hashing process of a task

Does this include devDependencies of the root package.json? I share devDependencies across all the projects, so will an update of webpack at the root-level also invalidate the hashes for individual projects that don't individually depend on webpack?

Also, what about the config situation I described? i.e.:

• run app:build
• change the configuration file in webpack-config
• run app:build (should be invalidated, because webpack-config changed)

It's my understanding that even if I make app:build depend on the noop task from webpack-config, it will not rebuild, because package.json has not changed, and the version bump hasn't happened, because the dependent package version is the same (workspace:*).

[milesj]

Does this include devDependencies of the root package.json? I share devDependencies across all the projects, so will an update of webpack at the root-level also invalidate the hashes for individual projects that don't individually depend on webpack?

Edit: Actually I think it does work this way: https://github.com/moonrepo/moon/blob/master/crates/node/platform/src/actions/run_target.rs#L276 We include changes from the root and project package.jsons.

It does include root package.json changes, but only if the dependency is also in the project's package.json. So it would need to be in both files.

Relying on those deps "merely existing" is non-ideal and won't be supported by moon. You can workaround this by just having the root package.json been an input for the task (/package.json).

Also, what about the config situation I described?

It should invalidate because webpack.config.js (or other files) are an input of the noop task, and that counts as a file change, invalidating the task itself, any anything else downstream.