Scio streaming pipeline stuck in Bigtable lookup

[regadas]

Operation ongoing in step Read from Master-Index for at least 05m00s without outputting or completing 
in state finish at sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at 
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:412) at 
com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:90) at 
com.foo.BigTableLookupDoFn.waitForFutures(BigTableLookupDoFn.scala:36) 
at com.spotify.scio.transforms.BaseAsyncLookupDoFn.finishBundle(BaseAsyncLookupDoFn.java:184) at 
com.foo.bigtable.BigTableLookupDoFn$DoFnInvoker.invokeFinishBundle(Unknown 
Source)

[mrkm4ntr]

Every time I faced this issue with following message. Found 1 stale streams and cancelled them This is from watchdog of Bigtable client.

[RustedBones]

Looks this is an issue from the deprecated Bigtable client. As beam is moving away from the bigtable-client-core v1, we should probably update the BigtableDoFn on the hbase API

[RustedBones]

Potentially fixed by #5209. Will wait for confirmation before closing