TypeScript: rules_nodejs ts_project isn't working using shard workers

[mr-pascal]

trafficstars

Used Bazel version: bazel 4.2.2-homebrew Used bazel-buildfarm branch: main

---

The Issue It's not possible to use shard buildfarm workers to build TypeScript projects using the ts_project rule of the rules_nodejs repository. --> EACCES errors are popping up.

Expected behaviour TypeScript projects can be built by the shard worker.

How to reproduce:

1. Spin up Redis instance (docker run -p 6379:6379 --rm --name dRedis redis:6.2.6-alpine), which is needed for shard worker
2. Checkout bazel-buildfarm
3. Change the property real_input_directories in the file bazel-buildfarm/examples/shard-worker.config.example to "external/npm"
4. Start the shard buildfarm server -> bazel run //src/main/java/build/buildfarm:buildfarm-server $PWD/examples/shard-server.config.example
5. Start the shard buildfarm worker -> bazel run //src/main/java/build/buildfarm:buildfarm-shard-worker $PWD/examples/shard-worker.config.example
6. Checkout issue-demo-1, which is a demo repository for this issue
7. Navigate into issue-demo-1 and run bazel build //... or ? bazel build //src/app:appto start the compliation of the TypeScript app (when in doubt, runbazel clean` before, to have a clean state)
8. See the errors of the bazel build command (compare screenshot)

To verify that the bazel project itself is working, remove the build --remote_executor=grpc://localhost:8980 line from the .bazelrc file of the issue-demo-1 repository and run the bazel build command again.

---

After having a short chat with @werkt we assume that the real_input_directories of the shard worker config might not work as expected.

[luxe]

Thanks for making a test repo. That will make it easy for us to reproduce.
rules_nodejs is definitely more exotic than other rulesets.

@80degreeswest / @jacobmou I think you have a buildfarm cluster & a repo with rules_nodejs.

1. Do you actually build them remotely (or are they tagged no-remote)?
2. Do you have any relevant patches that might have enabled remote functionality?

[werkt]

Ok, some color here:

real_input_directories of external/npm does not fix this problem because that is not the intended location for the symlink creation

The attempted symlink create is tied to the runfiles of the action: bazel-out/host/bin/external/npm/@bazel/typescript/bin/ts_project_options_validator.sh.runfiles/my_workspace/node_modules, which will be slightly more difficult to arrange into an exception to the input directory read-only policy. I can give it a shot, but will make no promises.

Specifying link_input_directories: false does get us past this issue, nominally by creating all-writable execRoot-local dirs for this purpose.

The writability of input directories and the variety of scenarios around their behavior has been deemed a remote-execution-service implementation-defined detail, so there's nothing we can point the nodejs people to to indicate that they're in the wrong here. If they would only create the symlink as a part of the action's definition for input on the bazel client, they could avoid this specification and actually have properly computed action definitions.

[mr-pascal]

@werkt thanks so much for your investigation. 🙏🏻 🙏🏻 🙏🏻

Just verified on my machine, using the above issue-demo-1 repository mentioned in the original post.

As @werkt suggested I set link_input_directories: false in the shard-worker.config.example and it seems to work. The build happens on the worker. 🎉

Even though I don't know how much impact in terms of scalability for very big monorepos this has. I guess there is a very good reason for the link_input_directories to default to true? 🤔

@werkt not sure if it changes anything, but it seems like the property real_input_directories doesn't care if it is external or external/npm.

If they would only create the symlink as a part of the action's definition for input on the bazel client, they could avoid this specification and actually have properly computed action definitions.

@alexeagle @gregmagolan @mattem, I hope it's okay that I link you here, but according to the commit contributions, it seems you are the main maintainers of the rules_nodejs repository. Any feedback here from your side would be highly appreciated. 🙏🏻 🙏🏻 🙏🏻