scio
scio copied to clipboard
Deadlock in AsyncLookupDoFn
Some dead-lock have been observed on the GrpcDoFn with the following trace
Operation ongoing in step Perform Terminator request/parDo:1 for at least 10m00s without outputting or completing in state finish in thread pool-4-thread-19 with id 78
at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
at [email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at app//com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:561)
at app//com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:111)
at app//com.spotify.scio.transforms.FutureHandlers$Guava.waitForFutures(FutureHandlers.java:49)
at app//com.spotify.scio.transforms.BaseAsyncLookupDoFn.finishBundle(BaseAsyncLookupDoFn.java:199)
at app//com.spotify.scio.grpc.GrpcDoFn$DoFnInvoker.invokeFinishBundle(Unknown Source)
This indicates one for the future never completes, though the stub was configured with a deadline.
Potentially fixed by https://github.com/spotify/scio/pull/5209. Will wait for confirmation before closing