Cluster design

Open magick93 opened this issue 8 months ago • 1 comments

NOTE: The following is WIP.

Kubernetes Cluster Design

Component	Recommended Kubernetes Resource Type	Notes	Suggested Node Grouping
blockprint	Deployment (Pod)	Core service, stateless	Group with blockprint-bg, blockgauge
blockprint-bg	Deployment (Pod)	Background service, stateless	Group with blockprint, blockgauge
blockgauge	Deployment (Pod)	Monitoring/metrics service, stateless	Group with blockprint, blockprint-bg
caddy	DaemonSet	Reverse proxy/load balancer, needs to run on all nodes	Run on all nodes
blockprint-ui	Deployment (Pod)	Frontend web interface, stateless	Can run on any node
lighthouse (main)	StatefulSet	Primary consensus client, stateful	Group with EL client
Execution Layer (EL)	StatefulSet	Execution client (Nethermind/Geth), stateful	Group with lighthouse main
eleel	Deployment (Pod)	Middleware connecting CL and EL, stateless	Group with lighthouse and EL
Additional CL clients	StatefulSet (per client)	Secondary consensus clients, stateful	Run on worker nodes
blockdreamer	?	Produce blocks every slot	Run on worker nodes

Core Services Group:
- blockprint
- blockprint-bg
- blockgauge
- blockprint-ui
- caddy (as DaemonSet)
Blockchain Clients Group:
- lighthouse (main instance)
- Execution Layer client
- eleel
Worker Nodes:
- Additional consensus clients (lighthouse instances)
- (Optional) Additional execution clients if needed
- blockdreamer

StatefulSets for blockchain clients (lighthouse, EL) should use PersistentVolumes
Consider storage class with automatic expansion for blockchain data
Monitor disk usage closely (previous issues with Nethermind running out of disk)

Ensure low latency between:
- blockprint services and main lighthouse instance
- lighthouse and its paired execution client
Ingress via caddy for web interfaces

Mar 27 '25 21:03 magick93

Added blockdreamer as a service that needs to run on the worker nodes.

https://github.com/blockprint-collective/blockdreamer

Apr 03 '25 05:04 michaelsproul