rules_scala icon indicating copy to clipboard operation
rules_scala copied to clipboard

implement multiplexer for scalac

Open wix-andriusb opened this issue 2 years ago • 3 comments

Description

Add multiplexer support for the scalac worker.

This creates a thread pool inside of the scalac worker and uses threads in order to use a singular process.

Motivation

Potential speed improvement and memory usage improvement because we use a single JVM process which warms up faster.

Implementation

The current implementation passes dedicated output streams to each worker which collect the output and then returns it to bazel.

wix-andriusb avatar Sep 27 '21 08:09 wix-andriusb

@simuons I created a dedicated MultiplexWorker.Interface and made only ScalacWorker utilize it, this means other workers can migrate independently to this new interface.

wix-andriusb avatar Nov 22 '21 12:11 wix-andriusb

Thanks @wix-andriusb for working on this I was starting to investigate how to do this change, @johnynek @simuons what is changes are outstanding before this can be approved? :smile:

JackTreble avatar Sep 09 '22 13:09 JackTreble

It did not make the compilation faster.

My guess is that it is bottlenecked by re-packing the jars.

wix-andriusb avatar Sep 09 '22 13:09 wix-andriusb

closing as stale, feel free to reopen

liucijus avatar Jan 31 '24 15:01 liucijus