turbo icon indicating copy to clipboard operation
turbo copied to clipboard

TurboRepo does not seem to rebuild (or even detect) "dependents" when using --filter and changing a sub-package

Open kylegwalsh opened this issue 2 years ago • 6 comments

What version of Turborepo are you using?

1.4.0

What package manager are you using / does the bug impact?

Yarn v2/v3 (node_modules linker only)

What operating system are you using?

Windows

Describe the Bug

I'm currently setting up TurboRepo alongside yarn workspaces and I'm having trouble getting it to rebuild our main app whenever one of the local packages that it relies on changes.

Here is an example of my file structure:

├── apps                   
│   └── web
├── packages              
│   ├── assets                 
│   ├── config                 
│   ├── design-system          
│   ├── hooks                  
│   └── utils              

Here is my turbo.json:

{
  "$schema": "https://turborepo.org/schema.json",
  "baseBranch": "origin/main",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": [".sst/**", ".build/**", ".expo/**"]
    }
  }
}

I would like to only rebuild packages/apps whenever they have changed in git, so I am using the --filter=[origin/main] option. I made a small change to the hooks package and expected it to need to rebuild hooks, hooks' dependencies, and web (because web has hooks as a dependency in its package.json). However, it only tries to rebuild hooks and hooks' dependencies.

To test, I am experimenting with the following command:

yarn turbo run build --filter=[origin/main] --dry-run

This prints out the following output:

@hlp/config#build
  Task          = build
  Package       = @hlp/config
  Hash          = 118d81f38208c721
  Directory     = packages\config
  Command       = <NONEXISTENT>
  Outputs       = .sst/**, .build/**, .expo/**
  Log File      = packages\config\.turbo\turbo-build.log
  Dependencies  =
  Dependendents = @hlp/hooks#build, @hlp/utils#build
@hlp/utils#build
  Task          = build
  Package       = @hlp/utils
  Hash          = 0c879c46fe4a9144
  Directory     = packages\utils
  Command       = <NONEXISTENT>
  Outputs       = .sst/**, .build/**, .expo/**
  Log File      = packages\utils\.turbo\turbo-build.log
  Dependencies  = @hlp/config#build
  Dependendents = @hlp/hooks#build
@hlp/hooks#build
  Task          = build
  Package       = @hlp/hooks
  Hash          = 4be940dedd5cc599
  Directory     = packages\hooks
  Command       = <NONEXISTENT>
  Outputs       = .sst/**, .build/**, .expo/**
  Log File      = packages\hooks\.turbo\turbo-build.log
  Dependencies  = @hlp/utils#build, @hlp/config#build
  Dependendents =

As you can see, it does not try to rebuild web. I also notice that web does not appear as a "dependent" of hooks when using this filter, however, it does appear if I remove the filter. Is there any way to trigger web to rebuild since its dependency changed?

Expected Behavior

I would have expected a change to a sub-package (like hooks) to trigger build in any package that lists it as a dependency (in this case web).

To Reproduce

  1. Create a yarn workspace with at least one app and one package.
  2. Install turborepo
  3. Add the following turbo.json
{
  "$schema": "https://turborepo.org/schema.json",
  "baseBranch": "origin/main",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": [".sst/**", ".build/**", ".expo/**"]
    }
  }
}
  1. Make the package a dependency of the app in the app's package.json
  2. Push everything to git
  3. Make a change to the package code
  4. Run the following command yarn turbo run build --filter=[origin/main] --dry-run

kylegwalsh avatar Aug 01 '22 18:08 kylegwalsh

You're looking for:

yarn turbo run build --filter=web...[origin/main] --dry-run

docs

tknickman avatar Aug 01 '22 18:08 tknickman

Thanks for the extremely speedy reply @tknickman

I just tried that command, but it results in no build tasks appearing. It doesn't seem to believe that web has changed despite one of its dependencies changing.

kylegwalsh avatar Aug 01 '22 18:08 kylegwalsh

Hmm ok I'll take a deeper look here and see if I can repro.

tknickman avatar Aug 01 '22 19:08 tknickman

Thanks, @tknickman. Other than that, everything has behaved exactly as expected!

kylegwalsh avatar Aug 01 '22 19:08 kylegwalsh

@tknickman Someone recommended that I try the following command it seemed to work. It's very close to the one you recommended, just without the web

yarn turbo run build --filter=...[origin/main] --dry-run

kylegwalsh avatar Aug 03 '22 15:08 kylegwalsh

Is there a recommended way to enforce deployment order via specific apps?

For example, I'd like my apps/backend to deploy before apps/web because web relies on output from the backend. I thought about using the following turbo.json:

{
  "$schema": "https://turborepo.org/schema.json",
  "baseBranch": "origin/main",
  "pipeline": {
    "deploy:ci": {
      "dependsOn": ["^deploy:ci"],
      "outputs": [".sst/**", ".build/**", ".expo/**"]
    }
  }
}

However, while this works if I add backend as a devDependency of web, it also results in backend always being rebuilt (even when nothing it's using changes). This is because if I change packages/hooks (which backend does not rely on), it will try to deploy packages/utils because hooks uses the utils package. This waterfalls and causes it to try to deploy backend because backend uses utils.

I'd also like to note that only the apps/* contain deploy:ci methods, so there is really no need for it to try to deploy changes to any package/* dependencies.

My end goal would look like the following:

  1. Change packages/hooks
  2. Detect change in packages/hooks and trigger deploy:ci for apps/web (which has hooks as a dependency)

Or

  1. Changes packages/utils
  2. Detect change in packages/utils and try to deploy both apps/backend and apps/web because they both rely on utils

I've tried replacing

 "dependsOn": ["^deploy:ci"],

with

 "dependsOn": [],

but then I cannot enforce backend deploys before web

kylegwalsh avatar Aug 03 '22 15:08 kylegwalsh

Looks like the original issue has been resolved so will close this one out!

anthonyshew avatar Feb 05 '24 04:02 anthonyshew