First-class support for virtual threads (OpenJDK's Project Loom)

[jwedel]

This issue is just a placeholder for future considerations regarding this topic.

Project Loom is an OpenJDK project that aims to provide support for Continuations and Fibers in the JVM natively. Here is a good article that explains the concepts and a couple of YouTube videos on this topic are available as well. There is also a public prototype available as well as at least one comprehensive git project example collection that implements a couple of different things (actors, TCP Proxy server, etc).

I wonder how some of the Spring projects could benefit in the end or if a new one could emerge. The first thing that comes to my mind it spring-web*. WebMvc is a rock solid and easy web implementation whereas WebFlux is a scalable reactive implementation. The latter comes with drawbacks in code complexity, readability and debuggability. Loom actually promises to allow scalability by spawning millions of Fibers and at the same time keeping the code readable by maintaining its sequential order.

I assume, some of the web server implementations like netty will eventually make use of Fibers, but does it make sense to apply the concepts to some Spring projects as well or will there be a clear boundary?

Have there been any considerations or ideas on how to leverage the power of Fibers and Continuations in the Spring eco-system?

[jwedel]

@wilkinsona pointed me to this article by @mp911de : https://paluch.biz/blog/182-experimenting-with-project-loom-eap-and-spring-webmvc.html

It shows significantly less memory usage at the same load. Especially the follow-up post is interesting, as it shows that missing synchronization support in Loom is preventing more utilization of Fibers as it blocks the carrier threads.

There is quite some work to do either in the JVM or/and in 3dr party libraries to fully leverage the potential.

But the example already shows, by switching to Loom in the background allows non-blocking applications like Flux does with the ease of the programming model of MVC.

[jhoeller]

Loom goes into preview as of JDK 19: https://twitter.com/OpenJDK/status/1519749893253861376

[jhoeller]

Virtual thread support is effectively a Spring Framework 6.1 topic, assuming that Project Loom may go out of preview for JDK 21 in September next year. For the time being, we aim to be compatible with virtual threads on a best-effort basis, supported by early adoption efforts around Spring Boot.

In early practice, this typically means a Tomcat setup with a virtual thread executor configured for its connector, and possibly a corresponding TaskExecutor for scheduling purposes, as illustrated by the following Spring Boot configuration bits (coming from Mark Paluch's early demos):

@Bean(TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME)
public AsyncTaskExecutor asyncTaskExecutor() {
	return new TaskExecutorAdapter(Executors.newVirtualThreadPerTaskExecutor());
}

@Bean
public TomcatProtocolHandlerCustomizer<?> protocolHandlerVirtualThreadExecutorCustomizer() {
	return protocolHandler -> {
		protocolHandler.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
	};
}

We are currently not aware of locking implications in the core framework or the Spring web stack. Many optimizations made it into our codebase over the years based on lock contention reports in highly concurrent production scenarios, so we do not expect I/O-bound synchronized usage (or any other forms of locking) to be part of any performance-critical code paths. All regular usage of synchronized in the Spring Framework codebase is effectively just around local data structures.

If we learn about concrete potential for virtual-thread oriented optimizations in the core framework, be it certain synchronized usage points or certain ThreadLocal usage, we'll try to roll corresponding refinements into the Spring Framework 6.0.x line already (as far as possible). First-class virtual thread configuration support, as far as sensible for core framework concerns such as TaskExecutor setup, is only expected for Spring Framework 6.1 at this point.

[jonnybecker]

Will RestTemplate then come out of maintenance mode or maybe the WebClient will get a non reactive API on top?

[vladimirfx]

I think is better to enhance Java HTTP client support instead of rehidrate RestTemplate.

[juliojgd]

Will RestTemplate then come out of maintenance mode or maybe the WebClient will get a non reactive API on top?

This is an interesting insight, as it can be generalized regarding the future of reactive programming as a whole in the Spring ecosystem.

I guess there is no much benefit in keeping two abstract execution models over the platform threads (one being ELG e.g. reactor-netty, and the other being virtual threads) both with the same aim at efficient use of resources avoiding blocked platform threads as far as possible.

I think that providing a non reactive API for a WebClient to allow use it with VT could be a way; and another approach (more long term) could be that the reactive approach had replaced its execution model by under-the-hood virtual threads.

Is there any planned (maybe in discussion right now) way of future action in the Spring ecosystem regardinf this, @jhoeller ?

T.I.A.

[jhoeller]

We have a ticket for Loom-driven revisiting of our WebClient: #29552 - however, we have yet to see where we are taking this.

As for RestTemplate, we'll make sure to test it for compatibility with virtual threads as well, not least of it all for existing RestTemplate-using application code.

[jhoeller]

We aim for tested compatibility with virtual threads in Spring Framework 6.1, against our current programming model. This is in preparation for Project Loom becoming a regular JVM feature but will proceed against the current Loom preview in JDK 19/20 for the time being. Some internal refinements may follow from this (e.g. locking optimizations) but API-wise we just intend to tweak some setup options, otherwise work with existing arrangements as far as possible. The goal is well-defined behavior for Spring Framework components when invoked within a virtual thread in any scenario, plus consistent ThreadFactory setup options to allow for specifying Thread.ofVirtual().factory() wherever necessary (this is largely possible already).

While our testing setups for virtual threads will likely run on JDK 21 EA, there is no immediate need for compiling against Loom-specific APIs for the purposes above. We intend to retain our JDK 17 build setup for the mainline, to be revisited when Loom goes out of preview (which would allow for compiling some explicit optional setup options for virtual threads then).

[anbusampath]

We intend to retain our JDK 17 build setup for the mainline, to be revisited when Loom goes out of preview (which would allow for compiling some explicit optional setup options for virtual threads then).

https://openjdk.org/jeps/8303683 - This JEP(still in Draft status) proposes to finalize Virtual Threads in JDK 21. It would be great to revisit Spring Framework 6.1 baseline as JDK 21 with Loom support.

[jhoeller]

Indeed, we expect Virtual Threads to become GA in JDK 21 now: https://openjdk.org/jeps/444

While our baseline will remain at JDK 17 level, we are considering a JDK 21 upgrade for the Spring Framework 6.1 build, providing first-class configuration options for virtual threads as well as virtual thread based integration tests, while still remaining compatible with JDK 17-20 at runtime.

[jhoeller]

We consider our virtual threads efforts as complete at the core framework level now. We expect some follow-up steps based on feedback, possibly before 6.1 GA but certainly in the 6.1.x line. The configuration options are not likely to get more sophisticated than #30241, however, there is always the chance for further fine-tuning of our synchronization and thread-local usage.