Use a virtual thread as the default blocking task executor if Project Loom is available

[ikhoon]

If Project Loom is activated on the JRE, we may use Thread.ofVirtual().factory() to create virtual threads for BlockingTaskExecutor instead. https://github.com/line/armeria/blob/d3c3811d7367e67ed4b7df47aa9fad5889948aed/core/src/main/java/com/linecorp/armeria/common/util/BlockingTaskExecutorBuilder.java#L131-L135

Virtual threads require a small number of threads. Users will have less work to tune thread counts with less memory footprint.

[jrhee17]

As a follow-up, I think running decorators from virtual threads would also be useful and a widely sought feature.

We may also want to consider either

1. Finding a way to set the workerGroup to a virtual thread executor so that decorators are also run from virtual threads
2. Providing a way to execute from a blocking executor from the HttpServerHandler (before the first decorator is triggered)

[Dogacel]

Considering Java 21 has been released pretty recently, I think this would be a cool feature to have now.

Does having virtual threads mean better performance for blocking tasks? Especially for blocking task executor?

As a follow-up, I think running decorators from virtual threads would also be useful and a widely sought feature.

Does that mean decorators will finally run outside the event loop while having a similar performance?

[jrhee17]

Hey thanks, we're definitely discussing designs on this at the moment.

Does having virtual threads mean better performance for blocking tasks? Especially for blocking task executor?

Currently, blockingTaskExecutor only schedules requests to the blocking executor for specific types of services (i.e. grpc, annotated). We would like to serve requests and callbacks from virtual threads entirely. (including decorators)

Does that mean decorators will finally run outside the event loop while having a similar performance?

That's the goal. Ideally, we will introduce a parameter which either accepts an (EventLoopGroup || virtual thread executor) so that users can choose between blocking and reactive style.

[vthacker]

Ideally, we will introduce a parameter which either accepts an (EventLoopGroup || virtual thread executor) so that users can choose between blocking and reactive style.

I guess that would be much bigger effort?

@ikhoon's proposal in the description is probably just switching to virtual threads from platform threads for the blocking task executor when applicable ( i.e if Armeria is running on JDK 21+) ?

[ikhoon]

I guess that would be much bigger effort?

Right, we need many changes to fully support virtual threads. Because we don't know when we will reach that goal, this issue would be a minimal effort to serve user services on the virtual threads.

[C0mbatwombat]

It would be a very useful improvement for us to make all processing possible via virtual threads. Not only for consistency and completeness of stack traces when using decorators (we are using a blocking task executor), but would also allow us to test our distributed system efficiently following this approach: https://jbaker.io/2022/05/09/project-loom-for-distributed-systems/

[chungonn]

Would "thread pinning" be a concern?

https://www.infoworld.com/article/3713220/java-virtual-threads-hit-with-pinning-issue.html

[ikhoon]

Thread pinning will be fixed in the next Java version. https://mail.openjdk.org/pipermail/loom-dev/2024-February/006433.html

We have been spending a lot of time implementing LINE's internal requirements such as xDS protocol and so on. We didn't have time to look into this issue in detail.

I knew many people are looking forward to using this feature. Let me try to upgrade the build JDK version to 21 and use virtual threads as an experimental blocking task executor in the next version.

[chungonn]

Thread pinning will be fixed in the next Java version. https://mail.openjdk.org/pipermail/loom-dev/2024-February/006433.html

We have been spending a lot of time implementing LINE's internal requirements such as xDS protocol and so on. We didn't have time to look into this issue in detail.

I knew many people are looking forward to using this feature. Let me try to upgrade the build JDK version to 21 and use virtual threads as an experimental blocking task executor in the next version.

By next Java version, which I suppose is Java 22 and that would not be an LTS.

I would suggest that you folks should wait for it to settle first before taking the leap. Without VirtualThread, the current Armeria is already doing more than what I can asked for.

[trustin]

I'll be working on this issue this quarter. Implementation ideas so far:

1. Allow a user to specify any ScheduledExecutor when building a service. Previously, only an EventLoopGroup was allowed:

   Server.builder()
         .route()
         ...
         .serviceWorkerGroup(myEventLoop)
         .build()
         ...
   

2. Provide an easy way to turn any Executor into ScheduledExecutor, allowing a user turn JDK's virtual thread executor into a ScheduledExecutor.
3. Provide a System flag that changes the default ScheduledExecutor implementation. If enabled, virtual thread based one will be used for both service workers and blocking task executors.

Some API changes:

• We add a new interface called ScheduledExecutor, which is a variant of BlockingTaskExecutor.
• BlockingTaskExecutor is deprecated in favor of ScheduledExecutor.
• A user can turn any Executor into a ScheduledExecutor.
• serviceWorkerGroup() is deprecated in favor of a new method (executor()?) that accepts a ScheduledExecutor.
• blockingTaskExecutor() will use a ScheduledExecutor rather than a BlockingTaskExecutor. (Maybe make ScheduledExecutor a subtype of BlockingTaskExecutor for compatibility?)

These are just ideas - any input would be appreciated. :bow: