OpenCensus stats not being incremented

[SanjayVas]

Possibly related to #7037

What version of gRPC-Java are you using?

1.48.1

What is your environment?

1. Linux (Debian Bookworm), JDK 11
2. Distroless Java 11 (gcr.io/distroless/java11-debian11:nonroot)

What did you expect to see?

zPages to be updated with OpenCensus stats from grpc-census.

What did you see instead?

/rpcz and /tracez are empty. /statz shows that views are registered, but none are incremented.

Steps to reproduce the bug

1. Start gRPC server
2. Send requests to gRPC server
3. Visit zPages in browser

Sample code (Kotlin):

    NettyServerBuilder.forPort(port)
      .apply {
        executor(executor)
        sslContext(sslContext)
        services.forEach { addService(it) }
      }
      .build()
      .start()

RpcViews.registerServerGrpcViews()
RpcViews.registerRealTimeMetricsViews()
ZPageHandlers.registerAllToHttpServer(httpServer)

httpServer.start()

[sanjaypujare]

@SanjayVas is this a regression? If yes, which is the earlier version where it was working?

[SanjayVas]

This is the first time I'm trying this, so I can't confirm what version it worked at previously. That said, I would assume it did work at some point as otherwise zPages would be rather useless.

[sanjaypujare]

You have 2 separate snippets of the code - I hope they are executed in the right order i.e. The RpcViews and ZPageHandlers registration first and then starting of the Netty server. Other than that, I can suggest the following:

• can you check if client side rpcs are reported (after changing to registerAllGrpcViews())
• try ZPageHandlers.startHttpServerAndRegisterAll(8000) instead of your own httpServer?
• try from Java instead of kotlin?

CC @ejona86 @jsuereth and @songy23 can you think of anything?

[SanjayVas]

I hope they are executed in the right order i.e. The RpcViews and ZPageHandlers registration first and then starting of the Netty server.

Ah, I indeed have them in the opposite order. Let me change that and get back to you.

[sanjaypujare]

Is it working now?

[SanjayVas]

Nope. Even with the registration and HttpServer starting up before the Netty gRPC server is built and started, I'm still not seeing anything in the results.

• try ZPageHandlers.startHttpServerAndRegisterAll(8000) instead of your own httpServer?

Looking at the code, this would do nothing. I'm doing exactly the same things as that method aside from registering the JVM shutdown hook, which is exactly the thing I want to avoid (I want to control server shutdown separately).

• try from Java instead of kotlin?

That would be a big change (this is hooked into other things in my build), and I'd be surprised if it would affect anything. You're welcome to try to reproduce my example in Java.

• can you check if client side rpcs are reported (after changing to registerAllGrpcViews())

I'll give this a shot.

[SanjayVas]

• can you check if client side rpcs are reported (after changing to registerAllGrpcViews())

No luck.

[sanjaypujare]

Couple of more suggestions:

• enable debug/trace logging and check for message output
• Replace ZPages with StackDriver and Google Cloud Monitoring and see if you see stats. If you do then the problem is with ZPages, if not then the problem is with OpenCensus.

[larry-safran]

@SanjayVas, were you able to try the last set of suggestions?

[SanjayVas]

We ended up switching to a different solution.