Vlad Lazar

@Noah-Kennedy thanks for looking into this. > what platform are you guys seeing this on? x86_64, Linux 5.10.0-14-amd64 Debian > did you do any recent bumps other than to tokio...

> @VladLazar looking at https://github.com/neondatabase/neon/pull/7048 I suspect there may be more to go through. @Noah-Kennedy Yeah, I did that initial check manually. I've scripted it and went through all `neondatabase/neon`...

@Noah-Kennedy don't mean to impose, but have you looked into this further by any chance? We've hit the same class of life-cycle assertion again yesterday on tokio 1.37.0 ([this one](https://github.com/tokio-rs/tokio/blob/9ed595767d01c400955122d276b34ab52b3a6aab/tokio/src/runtime/task/state.rs#L195-L200)...

I don't really understand what you mean by "using role: pageserver in the spans". Does enriching the trace on the collector using [OTTL](https://opentelemetry.io/docs/collector/transforming-telemetry/) solve your problem? You can enrich with...

> What I meant is that the hardcoded pageserver will now become a new attribute, like role, and service name will become the name of the "cluster" Understood! > Thanks!...

> We are at the early phase but I think it's possible to run multiple servers in the same bare metal host? Do you do that internally? If that's the...

> What about different pageservers running different shards for the same logical cluster? How do you annotate spans coming from different shards? Tenant id, shard id, and timeline id are...

> Internally, how do you annotate "cluster"? By cluster, I mean a Neon cluster, e.g., cluster_1, cluster_2, cluster_3... We do it by region. Idk what you mean by cluster here.

> Oh like we will deploy many Neon clusters. Each cluster is a combination of (safekeepers, pageservers, computes, storage-controller). I guess you must also deploy many clusters as well? Yeah,...

> What we are doing is just to publishing the spans to OTEL service, we don't have a sidecar running along side with the Neon components (as we don't control...