rules_kotlin icon indicating copy to clipboard operation
rules_kotlin copied to clipboard

Add initial test for multiplex worker corruption

Open restingbull opened this issue 2 years ago • 3 comments

restingbull avatar Jan 02 '22 22:01 restingbull

@restingbull I ran some tests in our codebase with the worker java implementation and wasn't able to reproduce the issue. I was however able to reproduce it almost instantly switching back to the coroutines implementation.

Bencodes avatar Jan 13 '22 23:01 Bencodes

There are a bunch of thread local variables in the compiler using the IR backend: https://github.com/JetBrains/kotlin/blob/54cca882797de3d7861fcddae0214ad802937669/compiler/ir/ir.tree/src/org/jetbrains/kotlin/ir/util/SymbolTable.kt#L360-L374

I'm not sure if something like that could be the source of this error, but it seems like it if switching to a stable thread caller vs coroutine that may switch threads - thread local seems like a likely cause of these issues. Maybe if you use a coroutine dispatch that always switches threads at yield points may be a good way to force and see if that is the issue? (although I'm not sure how to do that off the top of my head)

rockwotj avatar Jan 17 '22 05:01 rockwotj

Looks like you could do that with a dispatcher that always returns true for isDispatchNeeded and then have one that for dispatch creates a new thread for every task...

Docs: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/is-dispatch-needed.html https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-dispatcher/dispatch.html

Alternatively I wonder if you ran on a dispatcher with only a single thread if you wouldn't see corruption...

EDIT:

That doesn't make a ton of sense because the compiler is called synchronous without a coroutine context, It could also be that the default dispatchers are being reused in the kotlin compiler, leading to state being (reused) potentially when it shouldn't?

rockwotj avatar Jan 17 '22 05:01 rockwotj