java.lang.NullPointerException: Cannot invoke "build.buildfarm.common.redis.RedisClient.run

[80degreeswest]

`SEVERE ] build.buildfarm.server.ByteStreamService$1 onError - error reading ae1387dea0354326904444ded3f58bf432b728be/208709468(shard) at offset 0 after 2335 responses and 153026560 bytes of content java.lang.NullPointerException: Cannot invoke "build.buildfarm.common.redis.RedisClient.run(java.util.function.Consumer)" because "client" is null at build.buildfarm.instance.shard.JedisCasWorkerMap.adjust(JedisCasWorkerMap.java:76) at build.buildfarm.instance.shard.RedisShardBackplane.adjustBlobLocations(RedisShardBackplane.java:953) at build.buildfarm.instance.shard.Util.lambda$correctMissingBlob$1(Util.java:104) at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:242) at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:232) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.SettableFuture.set(SettableFuture.java:47) at build.buildfarm.instance.shard.Util$1.complete(Util.java:129) at build.buildfarm.instance.shard.Util$1.onSuccess(Util.java:142) at build.buildfarm.instance.shard.Util$1.onSuccess(Util.java:126) at build.buildfarm.instance.shard.Util$2.onSuccess(Util.java:164) at build.buildfarm.instance.shard.Util$2.onSuccess(Util.java:161) at build.buildfarm.instance.shard.Util$3.onSuccess(Util.java:197) at build.buildfarm.instance.shard.Util$3.onSuccess(Util.java:188) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.setResult(AbstractTransformFuture.java:247) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:163) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at io.grpc.stub.ClientCalls$GrpcFuture.set(ClientCalls.java:558) at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:531) at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553) at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718) at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) Exception in thread "Thread-59392" java.lang.NullPointerException: Cannot invoke "build.buildfarm.common.redis.RedisClient.run(java.util.function.Consumer)" because "this.client" is null at build.buildfarm.instance.shard.RedisShardBackplane.queueing(RedisShardBackplane.java:1340) at build.buildfarm.instance.shard.ShardInstance$1.lambda$iterate$0(ShardInstance.java:375) at build.buildfarm.common.Poller$ActivePoller.run(Poller.java:74) at java.base/java.lang.Thread.run(Thread.java:832) Exception in thread "Thread-59491" java.lang.NullPointerException: Cannot invoke "build.buildfarm.common.redis.RedisClient.run(java.util.function.Consumer)" because "this.client" is null at build.buildfarm.instance.shard.RedisShardBackplane.queueing(RedisShardBackplane.java:1340) at build.buildfarm.instance.shard.ShardInstance$1.lambda$iterate$0(ShardInstance.java:375) at build.buildfarm.common.Poller$ActivePoller.run(Poller.java:74) at java.base/java.lang.Thread.run(Thread.java:832) Exception in thread "Thread-59495" java.lang.NullPointerException: Cannot invoke "build.buildfarm.common.redis.RedisClient.run(java.util.function.Consumer)" because "this.client" is null at build.buildfarm.instance.shard.RedisShardBackplane.queueing(RedisShardBackplane.java:1340) at build.buildfarm.instance.shard.ShardInstance$1.lambda$iterate$0(ShardInstance.java:375) at build.buildfarm.common.Poller$ActivePoller.run(Poller.java:74) at java.base/java.lang.Thread.run(Thread.java:832) Exception in thread "Thread-59496" java.lang.NullPointerException: Cannot invoke "build.buildfarm.common.redis.RedisClient.run(java.util.function.Consumer)" because "this.client" is null at build.buildfarm.instance.shard.RedisShardBackplane.queueing(RedisShardBackplane.java:1340) at build.buildfarm.instance.shard.ShardInstance$1.lambda$iterate$0(ShardInstance.java:375) at build.buildfarm.common.Poller$ActivePoller.run(Poller.java:74) at java.base/java.lang.Thread.run(Thread.java:832) New

`

[werkt]

There are two possible circumstances for the client member being null, both involve the client not being 'started'. I believe this to be a race condition with a transform for a correctMissingBlob - this work should be accounted for (as in, is not) in the shutdown procedures for a client. Either you've managed to call correctMissingBlob before calling start() on the RedisShardBackplane, or (much more likely), it's been called after a stop() has been invoked. Both should have log messages either missing (and happening immediately) or (again in the much more likely case), messages indicating a shutdown of the instance has begun.