tezos-k8s

tezos-k8s copied to clipboard

Background: Currently, when deploying Tezos nodes, a headless service is created for each statefulset. From a discussion we had, our rationale behind possibly eliminating the headless service for public chain node sts's was because the nodes don't require individual DNS records; they do not rely on each other. However for private chains, nodes need to bootstrap together and utilize DNS records created by the headless service.

After further investigation, It's evident that even public chain nodes could require the headless services, for things such as indexers. Indexers target specific Tezos node pods. Using multiple nodes isn't ideal since different nodes can possess different states.

Required Action:

1. This discussion came up due to confusion of which service to forward ingress RPC traffic to. We need clear documentation and in-code comments emphasizing that for RPC traffic targeting, the tezos-node-rpc service should be preferred over headless services.
2. More instructions should be provided to configure label selectors for the tezos-node-rpc service in values.yaml allowing users to select only a subset of nodes, if needed, for RPC traffic. By default all nodes get RPC traffic forwarded to them by the service.

Relevant snippets: https://github.com/oxheadalpha/tezos-k8s/blob/5ec120a2e2390a6ea259be995bcd53ccf86b3d6b/charts/tezos/values.yaml#L272-L280 https://github.com/oxheadalpha/tezos-k8s/blob/5ec120a2e2390a6ea259be995bcd53ccf86b3d6b/charts/tezos/templates/static.yaml#L1-L34

"After further investigation, It's evident that even public chain nodes could require the headless services, for things such as indexers. Indexers target specific Tezos node pods. Using multiple nodes isn't ideal since different nodes can possess different states."

Can you elaborate on this ? I don't think this applies, at least for tzkt there is mechanism to handle 404 blocks with a retry, and it works fine with different rpcs behind a loadbalancer, they are aware of this since the beginning of the development I believe.

It would be safer to index a single node vs creating the index from multiple different sources. Because each node could theoretically have different state (as far as i understand).

I saw your issue https://github.com/baking-bad/tzkt/issues/146 about the 404s, but reorgs and nodes with diff state is still a possibility.

I see that tzkt has a lag option to handle potential reorgs: See https://github.com/baking-bad/tzkt#chain-reorgs-and-indexing-lag and https://github.com/baking-bad/tzkt/pull/88.

We also support BCD indexer and I'm not aware if it handles nodes reorgs.

Ok, thanks for the clarification!