openwhisk-runtime-java icon indicating copy to clipboard operation
openwhisk-runtime-java copied to clipboard

Published 20 hours ago • verified publisher icon apache

Support action-level concurrency in Java runtime.

Open Scusemua opened this issue 3 years ago • 3 comments

This change would enable support for action-level concurrency in the Java OpenWhisk runtime. The design is roughly modeled after the action-level concurrency support of the NodeJS environments. The Java Proxy checks the __OW_ALLOW_CONCURRENT environment variable to determine whether support should be enabled or not.

This does not address the action-level concurrency limit, and I imagine the values passed to the ThreadPoolExecutor will need to be changed for this to be used within OpenWhisk. I am not sure if the action-level concurrency configured for the action should affect the underlying ThreadPoolExecutor object or not, though it seems like it should.

At the very least, this PR serves as a starting point for evaluating whether the Java runtime can support action-level concurrency in this manner. For what it's worth, I have been using this modified runtime in personal projects for several weeks.

Scusemua avatar Dec 14 '21 21:12 Scusemua

Not extensively. We could probably create a unit test in which we have a single container serve multiple requests simultaneously and watch for this by checking the value of a variable that gets incremented for each concurrent request or something like that.

Scusemua avatar Sep 06 '22 16:09 Scusemua

Hey Ben I replied on your slack thread and didn't realize you had an open pr for this will take a look

bdoyle0182 avatar Sep 06 '22 16:09 bdoyle0182

On top of @rabbah 's point, the action concurrency feature in openwhisk is still fairly new and experimental. It adds an additional level of capacity planning required on top of the user correctly allocating an appropriate amount of memory to the function to know how many executions an instance can support. However this will be really hard for the user to test in practice as there's no guarantee the function execution will go to the same container or really any way for the user to verify that they ran things concurrently within the same container without writing special code to recognize it in their function.

Only a few that I know of are leveraging this system feature, I wonder if anyone has attempted to solve this or thought about how a user can deterministically stress test their function. Not a question for this pr, but we can move this to one of our generic forums like slack or the mailing list

bdoyle0182 avatar Sep 06 '22 17:09 bdoyle0182