optimism
optimism copied to clipboard
ethereum-optimism
Kustomize Instructions are broken, include nonstandard k8s CRD's for prometheus
Describe the bug
The kustomize templates at:
https://github.com/ethereum-optimism/optimism/tree/develop/infra/op-replica/kustomize
Include in the instructions to use a "configmaps" dir that doesn't exist under the "Building" header:
kubectl diff -k ./bases/configmaps/
and
kustomize build ./bases/configmaps/
After some guessing, I assume that I am supposed to run:
kustomize overlays/kovan-replica-0-5-15/
?
Not sure what the "public-overlays" folder is supposed to be for (not in readme)
But the template it produces includes references to ServiceMonitor and PodMonitor which I assume are from a prometheus operator that I was unaware was a requirement.
There also is some code referring to a proxy which does not appear to be documented or necessary.
Expected behavior
Really all I want is:
A simple Deployment That includes the l2geth and data-transport-layer images in the same pod with a
PersistentVolumeClaim for both their data directories, and a
Service for the l2geth RPC and the data-transport-layer with a parameter to fill in for the URL of the Ethereum RPC server it should use for sync.
I can orchestrate any monitoring using my existing processes.
This monitoring and waiter code that's mixed in makes a lot of assumptions about the k8s environment that may not be true, and obfuscates how exactly these 2 components interact with each other.
Yeah, the provided kustomizations are just examples of replica deployments to k8s that we use to deploy.
I'll update the base resources with basic deployment example.
any update on this? the instructions to run a full node are a rat's nest of indecipherable config vars that seem to have no purpose.
I made my own using regular old YAML. I found it very frustrating to decipher what the complete set of vars I would need for the dtl and l2geth were, so if I missed any, or any of these can be decrufted, let me know. This uses the AWS EBS persistent volume plugin, but that can be defined to whatever storage class you'd like.
In my setup I am using AWS Ingress templates which I have left out of this snippet to avoid adding tech-specific items to the template. I don't really understand what the "block signer key" is for, or why I need to add it so let me know if that's some dev/test param I can get rid of.
apiVersion: apps/v1
kind: Deployment
metadata:
name: optimism-mainnet
labels:
app: optimism-mainnet
spec:
replicas: 1
selector:
matchLabels:
app: optimism-mainnet
strategy:
type: Recreate
template:
metadata:
labels:
app: optimism-mainnet
spec:
volumes:
- name: optimism-dtl-data
persistentVolumeClaim:
claimName: optimism-dtl-data
- name: optimism-l2geth-replica-data
persistentVolumeClaim:
claimName: optimism-l2geth-replica-data
containers:
- name: optimism-data-transport-layer
image: ethereumoptimism/data-transport-layer:0.5.42
volumeMounts:
- name: optimism-dtl-data
mountPath: /data
envFrom:
- configMapRef:
name: optimism-dtl-env-vars
ports:
- containerPort: 7878
name: http-dtl-rpc
protocol: TCP
resources:
requests:
memory: "2000Mi"
cpu: "2"
limits:
memory: "2500Mi"
cpu: "2"
- name: optimism-l2geth-replica
image: ethereumoptimism/l2geth:0.5.23
command:
- geth
- --datadir=$(DATADIR)
- --password=$(DATADIR)/password
- --allow-insecure-unlock
- --unlock=$(BLOCK_SIGNER_ADDRESS)
- --mine
- --miner.etherbase=$(BLOCK_SIGNER_ADDRESS)
volumeMounts:
- name: optimism-l2geth-replica-data
mountPath: /data
envFrom:
- configMapRef:
name: optimism-l2geth-env-vars
ports:
- containerPort: 8545
name: http-l2geth-rpc
protocol: TCP
- containerPort: 8546
name: http-l2geth-ws
protocol: TCP
resources:
requests:
memory: "24000Mi"
cpu: "5"
limits:
memory: "25000Mi"
cpu: "5"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: optimism-l2geth-replica-data
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp3 # aws value -- change this if you arent using aws
resources:
requests:
storage: 3500Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: optimism-dtl-data
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp3 # aws value -- change this if you arent using aws
resources:
requests:
storage: 1500Gi
---
apiVersion: v1
kind: Service
metadata:
name: optimism-mainnet-l2geth
labels:
app: optimism-mainnet-l2geth
spec:
type: NodePort
selector:
app: optimism-mainnet
ports:
- name: http-l2geth-rpc
port: 8545
protocol: TCP
- name: http-l2geth-ws
port: 8546
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: optimism-mainnet-dtl
labels:
app: optimism-mainnet-dtl
spec:
type: NodePort
selector:
app: optimism-mainnet
ports:
- name: http-dtl-rpc
port: 7878
protocol: TCP
---
apiVersion: v1
kind: ConfigMap
metadata:
name: optimism-l2geth-env-vars
data:
CHAIN_ID: "10"
DATADIR: "/data"
NETWORK_ID: "10"
NO_DISCOVER: "true"
NO_USB: "true"
GASPRICE: "0"
GCMODE: "archive"
BLOCK_SIGNER_ADDRESS: "0x00000398232E2064F896018496b4b44b3D62751F"
BLOCK_SIGNER_PRIVATE_KEY: "6587ae678cf4fc9a33000cdbf9f35226b71dcc6a4684a31203241f9bcfd55d27"
BLOCK_SIGNER_PRIVATE_KEY_PASSWORD: "pwd"
ETH1_CTC_DEPLOYMENT_HEIGHT: "13596466"
ETH1_L1_FEE_WALLET_ADDRESS: "0x391716d440c151c42cdf1c95c1d83a5427bca52c"
ETH1_L1_CROSS_DOMAIN_MESSENGER_ADDRESS: "0x25ace71c97B33Cc4729CF772ae268934F7ab5fA1"
ETH1_L1_STANDARD_BRIDGE_ADDRESS: "0x99C9fc46f92E8a1c0deC1b1747d010903E884bE1"
ETH1_SYNC_SERVICE_ENABLE: "true"
L2GETH_GENESIS_URL: "https://storage.googleapis.com/optimism/mainnet/genesis-berlin.json"
L2GETH_GENESIS_HASH: "0x106b0a3247ca54714381b1109e82cc6b7e32fd79ae56fbcc2e7b1541122f84ea"
L2GETH_BERLIN_ACTIVATION_HEIGHT: "3950000"
ROLLUP_ADDRESS_MANAGER_OWNER_ADDRESS: "0x9BA6e03D8B90dE867373Db8cF1A58d2F7F006b3A"
ROLLUP_BACKEND: "l1"
ROLLUP_CLIENT_HTTP: "http://optimism-mainnet-dtl.<k8s namespace>:7878"
ROLLUP_DISABLE_TRANSFERS: "false"
ROLLUP_ENABLE_L2_GAS_POLLING: "false"
ROLLUP_GAS_PRICE_ORACLE_OWNER_ADDRESS: "0x648E3e8101BFaB7bf5997Bd007Fb473786019159"
ROLLUP_MAX_CALLDATA_SIZE: "40000"
ROLLUP_POLL_INTERVAL_FLAG: "500ms"
ROLLUP_SYNC_SERVICE_ENABLE: "true"
ROLLUP_TIMESTAMP_REFRESH: "5m"
ROLLUP_VERIFIER_ENABLE: "true"
RETRIES: "60"
ETH1_HTTP: "http://<k8s service for eth>:8545"
RPC_ADDR: "0.0.0.0"
RPC_API: "eth,rollup,net,web3,debug"
RPC_CORS_DOMAIN: "*"
RPC_ENABLE: "true"
RPC_PORT: "8545"
RPC_VHOSTS: "*"
TARGET_GAS_LIMIT: "15000000"
USING_OVM: "true"
WS_ADDR: "0.0.0.0"
WS_API: "eth,rollup,net,web3,debug"
WS_ORIGINS: "*"
WS: "true"
IPC_DISABLE: "true"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: optimism-dtl-env-vars
data:
DATA_TRANSPORT_LAYER__ADDRESS_MANAGER: "0xdE1FCfB0851916CA5101820A69b13a4E276bd81F"
DATA_TRANSPORT_LAYER__CONFIRMATIONS: "0"
DATA_TRANSPORT_LAYER__DANGEROUSLY_CATCH_ALL_ERRORS: "true"
DATA_TRANSPORT_LAYER__DB_PATH: "/data"
DATA_TRANSPORT_LAYER__DEFAULT_BACKEND: "l1"
DATA_TRANSPORT_LAYER__ENABLE_METRICS: "true"
DATA_TRANSPORT_LAYER__ETH_NETWORK_NAME: "mainnet"
DATA_TRANSPORT_LAYER__L1_GAS_PRICE_BACKEND: "l1"
DATA_TRANSPORT_LAYER__L1_START_HEIGHT: "13596466"
DATA_TRANSPORT_LAYER__L2_CHAIN_ID: "10"
DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT: "http://<k8s service for eth>:8545"
DATA_TRANSPORT_LAYER__L2_RPC_ENDPOINT: "http://optimism-mainnet-l2geth.<k8s namespace>:8545"
DATA_TRANSPORT_LAYER__LOGS_PER_POLLING_INTERVAL: "2000"
DATA_TRANSPORT_LAYER__NODE_ENV: "production"
DATA_TRANSPORT_LAYER__POLLING_INTERVAL: "100"
DATA_TRANSPORT_LAYER__SENTRY_TRACE_RATE: "0.05"
DATA_TRANSPORT_LAYER__SERVER_HOSTNAME: "0.0.0.0"
DATA_TRANSPORT_LAYER__SERVER_PORT: "7878"
DATA_TRANSPORT_LAYER__SYNC_FROM_L1: "true"
DATA_TRANSPORT_LAYER__SYNC_FROM_L2: "false"
DATA_TRANSPORT_LAYER__TRANSACTIONS_PER_POLLING_INTERVAL: "1000"
I didn't write the Kustomize instructions so I can't help much there, but I did put together a minimal docker setup here: https://github.com/smartcontracts/simple-optimism-node
Even if you don't want to run docker directly, you can probably use this to figure out exactly which environment variables are necessary and which aren't. Anything not listed in the docker setup is not strictly necessary.
ah thanks -- I will compare to that and reconcile any differences..
It seems strange to have to define all these variables that don't really change in between environments in normal operation.. Could all these boilerplate values be put into the Dockerfile with the defaults, or bundled in a more transparent way?
for example, the only params I actually am changing for l2geth are:
DATADIR ROLLUP_CLIENT_HTTP ETH1_HTTP ROLLUP_BACKEND
and DTL: DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT DATA_TRANSPORT_LAYER__SYNC_FROM_L1 DATA_TRANSPORT_LAYER__SYNC_FROM_L2 DATA_TRANSPORT_LAYER__DB_PATH
I think most people will never change a lot of these kind of params for mainnet unless there was explicit instructions from the optimism team: ETH1_L1_CROSS_DOMAIN_MESSENGER_ADDRESS, ETH1_L1_STANDARD_BRIDGE_ADDRESS, L2GETH_GENESIS_HASH etc..
If there was just a param called like "OPTIMISM_NET_ENV" that i could set to "mainnet", then the startup scripts in geth.sh/dtl.sh could just source a baked-in file with all these static env values for a given environment (allowing for overrides), and I wouldn't need to worry about when new ones are added or old values are changed.
Yeah this is what I tried to do with simple-optimism-node. I agree that would be the ideal way to manage environment variables. We currently have lots of boilerplate that users don't actually need to modify themselves.