Persistent workers with js_run_binary

[mancusi]

Apologies for the lack of detail here but is it expected that passing execution_requirements = {"supports-workers": "1"} to create a persistent worker from a js_run_binary would work?

I'm attempted to migrate a persistent worker created from a nodejs_binary and a custom rule to js_binary and js_run_binary but have noticed that the worker simply hangs indefinitely. The outputs are being produced and the build succeeds if it crashes (such as throwing an error on a timeout).

Running the worker in standalone mode produces a proper output without any changes to the tool and the worker works under the old setup.

The worker uses @bazel/worker to interface with the bazel worker protocol. I'm not sure if this is something the team has attempted or not or even expects to work. Do you know of any examples of a working persistent worker run with js_run_binary?

[gregmagolan]

It is not expected to work unless the underlying tool that js_run_binary supports worker mode. js_run_binary is a generic wrapper around any js_binary tool so it has no knowledge of whether or not the underlying tool supports worker mode.

You can read more about how to implement persistent workers here https://bazel.build/remote/creating.

We have a few custom rules that support worker mode,

• ts_project: https://github.com/aspect-build/rules_ts/blob/78b290e8b9f18210b5bd66bb8f543d3890229d38/ts/private/ts_project.bzl#L43
• webpack_bundle: https://github.com/aspect-build/rules_webpack/blob/766693b940f01e11a9d66fff72cb54a142f8fda6/webpack/private/webpack_bundle.bzl#L207

There is also a helper @bazel/worker npm package that handles some of the boilerplate of making a tool work with persistent worker mode, https://github.com/bazelbuild/rules_nodejs/blob/stable/packages/worker/README.md

[gregmagolan]

Oh I missed that bottom paragraph that you are already using @bazel/worker. Do you have the code up somewhere we can take a look at? I'm curious about where it is going wrong.

My guess is that this could work with js_run_binary if we added a supports_workers attribute and use a params file when that was set to True

    if ctx.attr.supports_workers:
        # Set to use a multiline param-file for worker mode
        arguments.use_param_file("@%s", use_always = True)
        arguments.set_param_file_format("multiline")

[mancusi]

I created a repository here which demostrates the issues I'm running into. The worker is here and can be invoked with bazel build //src:styles.

This build currently works but only because I'm intentionally crashing the worker here. If that line is removed then the worker simply hangs forever.

Quick Links

• worker
• usage

Hopefully this helps provide some insight into my issue

[mancusi]

It's entirely possibly that my worker or the macro is misconfigured somehow but the fact that I'm using @bazel/worker under the hood and the fact the worker.js creates the expected files when generated from the command line (node src/css_modules/worker.js src styles.css styles.css.js styles.css.module.css) leads me to believe this is an incompatibility with js_run_binary, persistent workers, or @bazel/worker.