sei-chain icon indicating copy to clipboard operation
sei-chain copied to clipboard

[BUG] error="fromProto: validatorSet proposer error: nil validator" closerError=""

Open changsongl opened this issue 1 year ago • 10 comments

Seid version v3.3.2

Chain ID atlantic-2

Describe the bug Can't start the node. It is a old bug that is reported by other issues.

https://github.com/sei-protocol/sei-chain/issues/575 https://github.com/sei-protocol/sei-chain/issues/1072

To Reproduce Follow the instruction to build the full node of testnet. seid start will show this error.

Expected behavior My node starts to sync the blocks.

Additional context Seed mode works, but full mode doesn't.

changsongl avatar Dec 20 '23 10:12 changsongl

hi @changsongl 👋 Could you provide the full console output when you encounter this issue please?

In the meantime, you could try these steps:

  1. Some RPC providers have seen instability in their systems recently. You could try starting your node again in a few hours, or tomorrow.
  2. You could double check that you have set your configs correctly here https://docs.sei.io/full-node/run-a-sei-node/join-a-network#update-configurations
  3. You could try using a different RPC node provider - theres a list here https://docs.sei.io/develop/resources
  4. You could try upgrading to seid v3.3.3 https://github.com/sei-protocol/sei-chain/releases/tag/v3.3.3

duncand0nuts avatar Dec 20 '23 15:12 duncand0nuts

hi @duncand0nuts Here is my full console output. I am just confused about why the error message is validator set, because I only want to set up a RPC node instead of a validator node.

bug.txt

changsongl avatar Dec 20 '23 16:12 changsongl

you may have a missing or incorrect genesis file. Try using this one here: https://github.com/sei-protocol/testnet/blob/main/atlantic-2/genesis.json

duncand0nuts avatar Dec 21 '23 10:12 duncand0nuts

Hello @duncand0nuts , I have the same problem on version 3.5.0 with chain-id pacific-1. I updated and checked configs according to docs. I use the genesis file from here. Besides that, I tried this genesis file which ended up with the following error:

root@Ubuntu-2204-jammy-amd64-base ~ # seid start
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x12e49ed]

goroutine 1 [running]:
github.com/cosmos/cosmos-sdk/server.StartCmd.func2(0xc000286e00?, {0x39bde60?, 0x0?, 0x0?})
	/root/go/pkg/mod/github.com/sei-protocol/[email protected]/server/start.go:173 +0x40d
github.com/spf13/cobra.(*Command).execute(0xc000002780, {0x39bde60, 0x0, 0x0})
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:872 +0x6aa
github.com/spf13/cobra.(*Command).ExecuteC(0xc000136c80)
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:990 +0x38d
github.com/spf13/cobra.(*Command).Execute(...)
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:918
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:911
github.com/cosmos/cosmos-sdk/server/cmd.Execute(0x38f4c28?, {0xc000610422, 0xa})
	/root/go/pkg/mod/github.com/sei-protocol/[email protected]/server/cmd/execute.go:35 +0x145
main.main()
	/root/sei-chain/cmd/seid/main.go:16 +0x2b

bdehri avatar Dec 28 '23 14:12 bdehri

Hello @duncand0nuts , I have the same problem on version 3.5.0 with chain-id pacific-1. I updated and checked configs according to docs. I use the genesis file from here. Besides that, I tried this genesis file which ended up with the following error:

root@Ubuntu-2204-jammy-amd64-base ~ # seid start
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x12e49ed]

goroutine 1 [running]:
github.com/cosmos/cosmos-sdk/server.StartCmd.func2(0xc000286e00?, {0x39bde60?, 0x0?, 0x0?})
	/root/go/pkg/mod/github.com/sei-protocol/[email protected]/server/start.go:173 +0x40d
github.com/spf13/cobra.(*Command).execute(0xc000002780, {0x39bde60, 0x0, 0x0})
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:872 +0x6aa
github.com/spf13/cobra.(*Command).ExecuteC(0xc000136c80)
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:990 +0x38d
github.com/spf13/cobra.(*Command).Execute(...)
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:918
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	/root/go/pkg/mod/github.com/spf13/[email protected]/command.go:911
github.com/cosmos/cosmos-sdk/server/cmd.Execute(0x38f4c28?, {0xc000610422, 0xa})
	/root/go/pkg/mod/github.com/sei-protocol/[email protected]/server/cmd/execute.go:35 +0x145
main.main()
	/root/sei-chain/cmd/seid/main.go:16 +0x2b

I got this working by using the genesis from the Sei Resources page, under sei-protocol: https://docs.sei.io/develop/resources https://raw.githubusercontent.com/sei-protocol/testnet/main/pacific-1/genesis.json

I still had to overwrite the genesis with the one from the above page, even after initialization. Also, I dunno why it's under a testnet directory on Sei's Github, but it is the pacific-1 mainnet genesis.

ymonye avatar Dec 30 '23 18:12 ymonye

Having the same issue. Seid version v3.5.0 (tested with 3.5.1 also)

Chain ID pacific-1

Sync state

STATE_SYNC_RPC=https://sei-state-sync-rpc.brocha.in:443
STATE_SYNC_PEER=94b63fddfc78230f51aeb7ac34b9fb86bd042a77@sei-state-sync.p2p.brocha.in:30615
LATEST_HEIGHT=$(curl -s $STATE_SYNC_RPC/block | jq -r .block.header.height)
SYNC_BLOCK_HEIGHT=$(($LATEST_HEIGHT - 60000))
SYNC_BLOCK_HASH=$(curl -s "$STATE_SYNC_RPC/block?height=$SYNC_BLOCK_HEIGHT" | jq -r .block_id.hash)

sed -i.bak -e "s|^enable *=.*|enable = true|" $HOME/.sei/config/config.toml
sed -i.bak -e "s|^rpc-servers *=.*|rpc-servers = \"$STATE_SYNC_RPC,$STATE_SYNC_RPC\"|" \
  $HOME/.sei/config/config.toml
sed -i.bak -e "s|^trust-height *=.*|trust-height = $SYNC_BLOCK_HEIGHT|" \
  $HOME/.sei/config/config.toml
sed -i.bak -e "s|^trust-hash *=.*|trust-hash = \"$SYNC_BLOCK_HASH\"|" \
  $HOME/.sei/config/config.toml
sed -i.bak -e "s|^bootstrap-peers *=.*|bootstrap-peers = \"$STATE_SYNC_PEER\"|" \
  $HOME/.sei/config/config.toml
sed -i.bak -e "s|^use-p2p *=.*|use-p2p = true|" $HOME/.sei/config/config.toml

Tried with genesis files from https://raw.githubusercontent.com/sei-protocol/testnet/main/pacific-1/genesis.json

seid start
2:51PM INF --tracing not passed in, tracing is not enabled
2:51PM INF Creating node metrics provider
2:51PM INF Starting Process
2:51PM INF starting node with ABCI Tendermint in-process
Error: error creating node: error="fromProto: validatorSet proposer error: nil validator" closerError=""

And genesis files from https://sei-state-sync-rpc.brocha.in/genesis

seid start
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x12e50cd]
...

zlace avatar Jan 10 '24 06:01 zlace

Made it work by resetting

rm -rf ~/.sei and running

seid init --chain-id "$CHAIN_ID" "$MONIKER" 
curl https://raw.githubusercontent.com/sei-protocol/testnet/main/pacific-1/genesis.json > $HOME/.sei/config/genesis.json
seid start

zlace avatar Jan 10 '24 07:01 zlace

Made it work by resetting

rm -rf ~/.sei and running

seid init --chain-id "$CHAIN_ID" "$MONIKER" 
curl https://raw.githubusercontent.com/sei-protocol/testnet/main/pacific-1/genesis.json > $HOME/.sei/config/genesis.json
seid start

This did help get past the initial error, thank you. Is your node able to connect to peers for statesync following that? Mine is not:

INF waiting for sufficient peers to start statesync duration=31m0.000200645s iters=18631 module=statesync peers=0 target=2

9:38PM ERR failed to send request for peers err="no available peers to send a PEX request to (retrying)"

MikeC-BC avatar Jan 11 '24 21:01 MikeC-BC

The segfault is tricky - lets try a full reinstall and see if the issue persists. it might actually be because the file doesnt have the .json extension?

If you are encountering state syncing and then removing stuff helped, next time take a look at the "clean up" steps here: https://docs.sei.io/full-node/run-a-sei-node/join-a-network#clean-up

Once you use the correct genesis file and perform a state sync, it should work

duncand0nuts avatar Jan 11 '24 21:01 duncand0nuts

The segfault is tricky - lets try a full reinstall and see if the issue persists. it might actually be because the file doesnt have the .json extension?

The segfault was caused by using an incorrect genesis file. The stack (/root/go/pkg/mod/github.com/sei-protocol/[email protected]/server/start.go:173) corresponds to the line here. Dumping the result from GenesisDocFromFile shows a nil genesisFile, and a non-nil err (silently dropped in the code). The error indicates an unmarshaling error.

Seems the genesis file fetched from RPC endpoint (e.g. https://sei-rpc.polkachu.com/genesis) has a different format, e.g. it has a genesis field at the top level ({"genesis":{"genesis_time": ...}}).

Once you use the correct genesis file and perform a state sync, it should work

Yep. Switching to the file in repo addresses the issue.

tbao2011 avatar Jan 30 '24 01:01 tbao2011