Feature request: Watch mode

[dobesv]

trafficstars

Describe the feature you'd like to request

It would be nice if turbo could run in "watch mode

Describe the solution you'd like

Turbo could watch all the files it uses to calculate the build hash for each package/workspace, and re-run the build if one of those files changes.

Describe alternatives you've considered

I think I could probably setup nodemon to run turbo when a file changes in a package, and then turbo will just only build packages with a changed file.

[b12k]

This functionality potentially may have a collision with existing scripts (for example in nestjs and nextjs apps).

For example: turbo run dev --watch where application dev npm script already has watch mode out of the box.

IMHO respective applications scripts is the right place for such things, not turbo.

[dobesv]

This functionality potentially may have a collision with existing scripts (for example in nestjs and nextjs apps). turbo run dev --watch where application dev npm script already has watch mode out of the box.

I think it's simple not to use the watch feature in that case, isn't it ?

IMHO respective applications scripts is the right place for such things, not turbo.

Seems a fair point. However, I think turbo is specially positioned to know which files should trigger a build and watch those files. Providing it as part of turbo would be a convenience.

[franco-roura]

I second the motion, you usually don't want to "dev" your apps until you have "built" your packages, spawning dev in apps&packages concurrently will cause the app not to find the built package in some scenarios.

On the other hand, adding "dependsOn": ["^dev"] to the turbo.json is not an option, daemon/watcher scripts don't finish, so the dependency is never satisfied and apps/*/dev never starts.

If Turbo had an inner watcher to trigger build or codegen scripts on file changes it would save an interesting amount of time and performance by preventing developers from manually maintaining a script like the following for each package:

{
"watch:someapp": "nodemon --watch 'packages/somepackage' -e ts --exec 'turbo run build --scope=somepackage'"
}

[hrasekj]

It would be great feature. For example, when building packages with swc, and still need typechecks between packages.

Example

My solution on monorepo i'm working on, with 20+ packages.

turbo.json

{
  "pipeline": {
    "build": {
      "dependsOn": ["types", "^build"],
      "outputs": ["dist/**", ".next/**"]
    },
    "types": {
      "cache": false,
      "outputs": ["typings/**"]
    },
    "watch": {
      "cache": false
    }
  }
}

root package.json

{
  "name": "root",
  "private": true,
  "scripts": {
    "build": "turbo run build",
    "watch": "turbo run watch --parallel"
  }
}

packages/my-helpers/package.json

{
  "name": "my-helpers",
  "scripts": {
    "types": "tsc --emitDeclarationsOnly",
    "watch": "chokidar 'src/**/*.js' 'src/**/*.ts' -c 'pnpm -wc exec turbo run types  --filter=...^my-helpers'",
  }
}

This solution is nice, because i can see, that changes in one package will affect another package in scope.

There are some drawbacks in this:

• can't cache types pipeline
• must write package name inside watch script
• output logs are not nice

my-helpers:watch: • Packages in scope: @admin/web
my-helpers:watch: • Running types in 5 packages
my-helpers:watch: • Running types in 5 packages
my-helpers:watch: @admin/web:types: cache miss, executing 25984f71faae504a
...

Proposition

As i see it. Would be nice something like this in turbo.

turbo.json

{
  "pipeline": {
    "build": {
      "dependsOn": ["types", "^build"],
      "outputs": ["dist/**", ".next/**"]
    },
    "types": {
      "outputs": ["typings/**"]
    }
  }
}

root package.json

{
  "name": "root",
  "private": true,
  "scripts": {
    "watch": "turbo run types --watch"
  }
}

packages/my-lib/package.json

{
  "name": "my-lib",
  "scripts": {
    "types": "tsc --emitDeclarationsOnly"
  },
  "turboWatch": [
    "src/**/*.js",
    "src/**/*.ts",
  ]
}

Watch mode, should:

• run types script every time, when files described by turboWatch changes
• update types cache, on change, if necessary

Then running build pipeline will have types already run from cache.

[irontitan76]

I think this functionality is almost necessary at scale. Rather than running countless processes to watch each dependency for a target, one process should watch all the files that are in the dependency graph. If a change is detected then rerun the script for the affected dependencies. Rush.js does something similar with its experimental watchForChanges property.

Obviously, this functionality wouldn't be needed where watching becomes redundant and that is up for the developer to decide. So, if a developer had scripts like the following, he or she would not need to (and really should not) run this watch functionality with dev.

"scripts": {
  "build": "tsc --build",
  "dev": "tsc --watch",
}

I'm brainstorming a few ways to solve this with what @hrasekj stated being a valid solution. At the present moment, I think we could add a watch boolean property to the pipeline configuration objects. If true, it watches the files defined as inputs (if present) in each of the packages, but it does this as a whole based on the graph, rather than as individual processes.

[georeith]

Big +1 for this.

We're using Jest for tests and its watch mode doesn't detect changes in transitive monorepo dependencies like Turbo does for its caching.

On an extra note: a lot of watch modes overwrite STDOUT instead of appending, combined with something like this https://github.com/vercel/turborepo/issues/219 and the option to overwrite STDOUT we could make watched output much easier to read.

[adam-coster]

This would be very helpful for places where native watchers fall short, which is pretty common for complex pipelines and inter-package dependencies.

For example, I have a monorepo that includes a package that must be run in a docker container. Getting that to work with the other packages is a pain -- the best way would be to have the container rebuild itself whenever the other dependencies change, so that it can re-copy those does into the container and relaunch it. If turbo had a watch mode that would be pretty straightforward. In the absence I that I have to do some shenanigans that don't auto-adapt to changes in the dependency graph, so both complexity and maintenance costs go way up.

I'd much rather run all of my scripts in non-watch mode and then have Turbo manage watching, since most tools that have built-in watch mode have extremely limited utility in the context of pipeline.

[ScottAwesome]

The litmus test for me on this is how wireit handles watching for changes. It works extremely well, and should be the model. Its pretty neat how they solved this on the tool level.

[awxalbert]

Same here, my scenario is when I develop Apollo GraphQL server, I may end up with a package.json like:

{
  "dev": "ts-node -r dotenv/config src/index.ts",
  "generate": "graphql-codegen"
}

Currently, I use nodemon and codegen watch mode to do this, which is really inconvenient and have problems. Really looking forward to having this feature.

[arimus]

Things can get tricky with build deps and triggering commands, which the workarounds provided above don't account for, so far as I can tell. Simple filesystem monitoring, without dependency graph context, are not solutions when the graph isn't simple / linear (e.g. A depends on B depends on C).

For example, a common scenario would be where an app depends on multiple libs that are all rebuilt. Even if you set up scripts to trigger once a dependency changes, the build order has to fully complete before a restart of the app should be triggered. Slightly more complex example with a single library at the base of those two libraries incorporated into a single app:

        ______ Lib A _____
       /                  \
Lib 0 -                    My App
       \______ Lib B _____/

• Lib 0 changed
• Lib 0 rebuilt
  • Lib A rebuild
    • changes detected, rebuild / restart My App
  • Lib B rebuild
    • changes detected, rebuild / restart My App

Of course you can put the de-bounce up really high to ensure the build of all the things happens before triggering the My App rebuild, but this is not a great solution for obvious reasons.

Turbo knows the entire build graph, so it would be fantastic if we could leverage that to only run some commands after turbo knows that all linked deps have built.

[dobesv]

Another question: if watch mode wasn't added in the core, does turbo export enough functionality as a go library to write a separate tool that reuses code from turbo to implement a watch mode? Maybe we could put up a bounty somewhere and someone can write another go app that wraps turbo with a watch mode.

[arimus]

Another question: if watch mode wasn't added in the core, does turbo export enough functionality as a go library to write a separate tool that reuses code from turbo to implement a watch mode? Maybe we could put up a bounty somewhere and someone can write another go app that wraps turbo with a watch mode.

Unless I'm mistaken, I don't think that turbo repo exports any functionality and there is no extensibility at the moment. That was one of the things that nx provided over turborepo. I agree that if there were some lifecycle plugin hooks for turbo, then that would require less to maintain in the core while still providing the necessary build context for separate tools. Even something as simple as hooks for the same events that are output to the console during build (cache replayed, cache miss / executing, build complete). Or barring that, executing registered commands for them as they occur.

File system watches alone won't work for many cases like my example above, unless you integrate the dependency graph as context. It still gets messy and complicated though. In the end, it'd be easier and more flexible for turbo (which already has the context) to trigger / notify at the right stages. Still stays simple and clean without a bunch of extra complexity.

[weyert]

I think they might be planning to use the (experimental) daemon for this; to allow script ability in Node.js by a socket connection using protobuffers

[scamden]

another +1. rush has this: https://rushjs.io/pages/advanced/watch_mode/ for inspo (oh just read https://github.com/vercel/turbo/issues/986#issuecomment-1104142740 but link could still be helpful for reference). Works really well if you want to build everything in watch mode and have your dev processes just responding to the compiled files (rather than the overhead of numerous compilers running simultaneously)

also didn't yall say this was up next like a while ago? https://turborepo.com/posts/turbo-0-4-0

[dobesv]

I would be willing to try working on this, if the team would be willing to accept the pull request.

[nathanhammond]

This is something we are considering as a possible future. (There is a reason the issue remains open.)

We're not opposed to accepting it from an external contributor as a way to have it bumped up in priority but do know that it's not a trivial effort and if undertaken now will need to be written in both Go and Rust.

[weyert]

Why in both? I thought you would use Go code as a library in Rust? To me, requiring both, massively increase the hurdles for contributions

[nathanhammond]

@weyert because we can't ship it in Rust right now with the state of the migration so it'd have to be Go first, and then Rust eventually.

[gajus]

We needed something that works today with Turborepo, so I wrote Turbowatch

https://github.com/gajus/turbowatch#why-not-turborepo

[dobesv]

Turbowatch

For a second I thought it was actually something specific to turborepo. It would be nice if turbowatch could parse turbo.json and automatically watch based on the inputs for each task and then run turbo as appropriate to the changed files/targets/projects

[AlexAegis]

@hrasekj earlier had an example on how they solve live type updates. I went with a different route that does not involve turborepo or running anything at all to get instant type feedback, but if you with to publish it, it does require the help of a build tool (So I made a vite plugin).

The idea is that in the source packageJson (the one checked into your git repo), you define exports like this:

{
	"exports": {
		".": {
			"types": "./src/index.ts",
			"import": "./dist/index.js",
			"require": "./dist/index.cjs"
		}
	},
}

And once I build it, the packageJson file that will be distributed (that ends up in the dist directory) will have these paths adjusted to:

This packageJson is practically invisible when the package is used locally, this is only used for distribution

{
	"exports": {
		".": {
			"types": "./index.d.ts",
			"import": "./index.js",
			"require": "./index.cjs"
		},
}

Being in a pnpm/npm workspace, my local packages are simply symlinked to eachothers node_modules folder so they only see the source packageJson file, telling typescript to watch the types directly from the source is an easy solution to solve live updates.

But this approach leaves a big problem behind: I still need to manually rebuild libraries that I use in applications. I can see the types as I write them, I just can't use them, and if I build an application in it's own dev watch mode, only that apps code is rebuilt. But as soon as I build the dependencies, the apps dev server sees that change and hot reloads my app. This is the last puzzle piece that I'm missing.

[marek-hanzal]

I've just want to add myself as a watcher as I've turborepo with quite a few packages and I've to manually switch witch ones should be watched as they're worked on. This is awfull, also starting tens of watches (2 per package: esbuild for JS and TSC for types) is incredibly heavy.

[gajus]

This is awfull, also starting tens of watches (2 per package: esbuild for JS and TSC for types) is incredibly heavy.

Please check https://github.com/gajus/turbowatch, and specifically the part about the motivation behind the project, as it addresses why I think it is unlikely that whatever the solution that Turborepo comes up with will be an optimal solution for everyone (due to the declarative nature of turbo.json).

The closest I've seen something that is reasonably close to a sane declarative API for this use case is https://github.com/google/wireit. However, even that would not have worked in our case due to a mixed bag of requirements (such as rebuilding and re-launching Docker containers as part of the dependency tree).

For what it is worth, we are very happy with the current setup (Turborepo + Turbowatch). However, I am also cognizant of the fact that our repo is probably hitting every imaginable edge case in terms of requirements, and that alternative solutions could be a lot more straightforward for those who manage to avoid our requirements.

[dobesv]

I've just want to add myself as a watcher

For future reference, these are the preferred way to follow / vote for a github item if you don't have a substantive comment:

• Subscribe button in the right sidebar
• Add a reaction to the issue description (e.g. a 👍 )

Hopefully you don't mind me giving this tip, just trying to be helpful!

[gajus]

Insights after developing Turbowatch. By sharing these insights, it is my intention to offer guidance that is helpful as we consider adding watch mode to turbo:

• Incorporate retry logic: In watch mode, failures are common, making the implementation of retry logic essential to ensure robust operation.
• Avoid combining multiple tools in watch mode: Combining tools such as tsc --watch and tsc-alias --watch leads to unexpected outcomes. I recommend using them in non-watch mode, as demonstrated in this example.
• Managing buildables and running services: Combining buildables (e.g., tsc and Docker build) with running services that require restarting or hot reloading can be challenging. We found success utilizing a mix of interruptible and persistent attributes (see Turbowatch documentation).
• Handling workspaces: Integrating the aforementioned components within a single workspace (as opposed to a dependency tree) can be particularly complex. Need to carefully think about which files different steps update and what expressions to use for triggering them, e.g. combining generate (buildable) and start-server (interruptible long running process). It cannot be one and the same script.
• Emphasize atomic changes for stability: For example, using rm -fr dist && tsc will trigger watch operations multiple times, causing initial failures due to the removal of assets and a subsequent delay before new assets become available.
• Optimize updates for Developer Experience (DX): To improve DX, update only what has changed. For instance, use an intermediate build directory and rsync to reduce the unnecessary reloads.
• Implement debounce: Related to the two points above, even if you try to make changes atomic, there is going to be some delay between operations (e.g. even rm -fr takes some time to run). Watch mode needs to have debounce built in to avoid re-running tasks immediately after every time that changes are detected. In practice, we found that a delay of 1 second provides a good balance of performance and avoiding unnecessary re-builds.
• Monitor dependency changes: Detecting changes in dependencies is generally not an issue as long as a convention is followed. We used the following expression for rebuilding a package when either the source or dependencies are updated. In the context of turbo, this should be less of an issue since tasks define inputs and outputs.
• Ensure graceful service termination: To avoid lingering Docker containers and hanging services, ensure services terminate gracefully. Unfortunately, Turbo does not wait for processes to exit gracefully (issue #4274), so we used turbowatch **/turbowatch.ts to start our services.
• Manage logs effectively: Reading logs from multiple workspaces in watch mode can be challenging.
  • Throttling logs to batch related logs is the most effective solution we discovered.
  • Using Roarr structured logs to make it easy to filter and manipulate logs.
  • Prefixing child process output with a randomly assigned ID to help debug the origin of logs.
• Abstract file watching backend: Through trial and error, I've learned that every existing file watching solution has issues, e.g. Watchman does not track symbolic links (#105), chokidar is failing to register file changes (#1240; for context, this made Vite HMR stop working #12459), fs.watch behavior is platform specific, etc. For Turbowatch this meant that we had to make the file watching backend swappable in the user-land. Turbowatch uses a combination of these options depending on the user's environment.
• Abstract script execution: Just running npm scripts is extremely limiting. We are already using zx internally to abstract bash scripting and it only made sense to make it available in Turbowatch.
• Consider how to combine watch mode with manual task execution: There are steps in our setup that need to run once, but then may need to be re-run during the lifetime of the watch mode, e.g. We have a script to download assets from Figma. It makes no sense to re-run this logic based on local file changes, but need to provide a way for users to re-run this script when they need to. In our case, we run it once before initiating Turbowatch (i.e. turbo generate && turbowatch) and then anyone can run it manually by using turbo generate, which will be picked up by Turbowarch. For Turborepo, this probably means that watch scripts just utilize dependsOn to run some tasks once before going into the watch mode.

[marek-hanzal]

Hello,

so @gajus I wanna call you out as I've moved my project with ~52 packages and other project with ~30 monorepo packages from weird combo concurently with esbuild & tsc to turbowatch aaaand I'm finally able to run the whole project with live reloads! Sometimes when a dependency changes I've to trigger by dummy file change, but I don't see it as a big deal, my flow is now much much smoother.

[gsoltis]

@gajus just want to chime in and say that this is awesome. Lots of great points and experience.

I can definitely sympathize with filewatching pains, I have similar experiences in that ecosystem.

I would definitely like to see watch mode integrated into turbo. As you note, there are a few things turbo needs to fix up as building blocks here, like better SIGINT handling and some log management.

I'd love to get your thoughts on other blockers that would be helpful to sort out as groundwork.

[gajus]

I'd love to get your thoughts on other blockers that would be helpful to sort out as groundwork.

I am not aware of other blockers per se.

Turbowatch + Turbo combination already works for our use case. It is only the question of how to embed these learnings into Turborepo API, which is mostly declarative at the moment.

Happy to hop on a call with Vercel team if you ever want to brainstorm about it.

[scamden]

My vote is to find a way to achieve this and stay declarative fwiw.

[DesignByOnyx]

There is a handy package out there called workspace-tools, built and maintained by Microsoft, and works for any popular monorepo: npm, yarn, pnpm, rush, lerna. I was able to use this tool to build a graph and then use turbowatch to do the work (thanks @gajus).

Dear turborepo: please expose the utils for getting the graph so we don't have to use 3rd party libraries. Thanks!

I wanted to post this to help move this conversation along because I believe we need to solve this in user land in order for it to be successfully implemented in turbo land. The script below takes a single argument for the package you want to watch. This can easily be expanded to multiple packages... but that's not my focus right now. I am posting this so others might be able to expand on it. Save this code to a file named watch.ts anywhere in your monorepo:

Install dependencies: npm i -D turbowatch tsx workspace-tools

Usage: tsx watch.ts <workspace_name> (you can use ts-node in place of tsx, or convert this to JS)

import path from 'path';
import { ChangeEvent, Expression, watch } from 'turbowatch';
import { getPackageInfos, createPackageGraph, getWorkspaceRoot, PackageGraph } from 'workspace-tools';

const cwd = process.cwd();
const workspaceRoot = getWorkspaceRoot(cwd);
const packageInfos = getPackageInfos(workspaceRoot);
const targetWorkspace = process.argv[2];

if (!targetWorkspace || !(targetWorkspace in packageInfos)) {
  throw new Error(
    `${targetWorkspace}" is not a valid workspace. Must pass a valid workspace name as it appears in package.json.`
  );
}

/** Gets all direct and transitive dependencies for a workspace */
const getAllDependencies = (target: string, graph: PackageGraph) => {
  return Array.from(
    graph.dependencies.reduce((acc, dep) => {
      if (dep.name === target) {
        acc.add(dep.dependency);
        getAllDependencies(dep.dependency, graph).forEach((val) => acc.add(val));
      }
      return acc;
    }, new Set<string>())
  );
};

const graph = createPackageGraph(packageInfos);
const dependencies = getAllDependencies(targetWorkspace, graph);
// path names will be in the format "packages/foo", "apps/bar" - for use in turbowatch expressions
const depPaths = dependencies.map((dep) => {
  const pkgInfo = packageInfos[dep];
  return path.dirname(pkgInfo.packageJsonPath).replace(`${workspaceRoot}/`, '');
});

const start = Date.now();
watch({
  project: workspaceRoot,
  triggers: [
    {
      expression: [
        'allof',
        ['not', ['anyof', ['dirname', 'node_modules'], ['dirname', 'dist'], ['dirname', 'out']]],
        ['anyof', ...depPaths.map((dir) => ['dirname', dir] satisfies Expression)],
        ['anyof', ['match', '*.ts', 'basename'], ['match', '*.js', 'basename'], ['match', '*.json', 'basename']],
      ],
      name: `${targetWorkspace}_deps`,
      interruptible: true,
      initialRun: true,
      persistent: true,
      onChange: async ({ spawn, files, first, abortSignal }: ChangeEvent) => {
        if (first) {
          console.log('Turbowatch started in', Date.now() - start, 'ms');
        } else {
          console.log('File change detected', files);
        }
        console.log('Building workspace dependencies');
        // build all of the dependencies, relying on turbo's cache to do the minimum necessary
        await spawn`turbo build --filter=${targetWorkspace}^...`;
        if (abortSignal.aborted) return;
        // start the workspace watch script
        console.log('Starting watch task');
        await spawn`turbo run watch --only --filter ${targetWorkspace}`;
      },
    },
  ],
});