Problem: query block json-rpc api occasionally fail

[yihuang]

System info: [Include Ethermint commit, operating system name, and other relevant details]

Steps to reproduce:

query block occasionally fail:

'code': -32000, 'message': 'rpc error: code = Unknown desc = rpc error: code = Internal desc = failed to obtain coinbase address: failed to retrieve validator from block proposer address tcrcvalcons1v0rpejs3vanaccqrx4xkez3mlkmpxnugj3ga55: validator does not exist: unknown request'

Expected behavior: should always work

Actual behavior: [What actually happened]

Additional info: [Include gist of relevant config, logs, etc.]

[VictorTrustyDev]

I am facing this issue also and have some more information about it Let's say about this DIFFUSION contract 0x3f75ceabCDfed1aCa03257Dc6Bdc0408E2b4b026, it was created at block 59500 so I query the contract name with context height 59500+1=59501, I got rpc error: code = Unknown desc = rpc error: code = Internal desc = failed to obtain coinbase address: failed to retrieve validator from block proposer address evmosvalcons1z0tmsqqlt725lmsxjhn2sm09jkj6zcvv4jze8s: validator does not exist: unknown request Weird is this is the NOMIC.ONE validator since block 2673107 and this validator was created at 2672608 by this transaction (different moniker name but same valoper) which is definately far away from 59501 and after I try again, ethclient.ContractCall still returns same error but the proposer became evmosvalcons1nsczfx3qr75f3anp4lklcanm585x7vwfuw3mt4 is Hanchon.live since block 486371, created at 461420

This is how I get contract name

slicedEvmAddr := evmAddr[2:]
contract := common.HexToAddress(evmAddr)
callMsg := ethereum.CallMsg{
	To:   &contract,
	Data: common.FromHex(fmt.Sprintf("%s000000000000000000000000%s", funcSelector, slicedEvmAddr)),
}
ctxHeightBi := big.NewInt(height + 1)
queryCtx := rpctypes.ContextWithHeight(ctxHeightBi.Int64())
resC, errC = cp.ethClient.CallContract(queryCtx, callMsg, ctxHeightBi)

(I already query successfully many contracts)

Let's refer to this error in ethermint project which is "failed to obtain coinbase address" and this code (ethermint also) we see the proposer from BlockHeader (proto Header) At first, I guess somehow the BlockHeader mistakenly becomes "current latest block" but there is another contract get name failed with same error but this validator which was stopped working for 2 days and I just query it today AND I realize my latest block on my archive node is ~3,770,000 which was 7 days ago (still syncing) so at this moment, I think the Proposer in this code somehow is Current proposer of Current state on the node

Anyone have any idea?

[VictorTrustyDev]

I think the Proposer in this code somehow is Current proposer of Current state on the node

It's totally correct. This is how I debug it:

• I have 2 servers, one to query data, one is node, ensure time on both servers are the same
• I print debug message when I call function get name of contract 0xd4949664cd82660aae99bedc034a0dea8a0bd517

{"level":"debug","context height":59141,"time":"2022-09-09T15:29:10Z","message":"NAME 0xd4949664cd82660aae99bedc034a0dea8a0bd517"} {"level":"debug","time":"2022-09-09T15:29:40Z","message":"FINISH name 0xd4949664cd82660aae99bedc034a0dea8a0bd517"} {"level":"error","contract":"0xd4949664cd82660aae99bedc034a0dea8a0bd517","error":"rpc error: code = Unknown desc = rpc error: code = Internal desc = failed to obtain coinbase address: failed to retrieve validator from block proposer address evmosvalcons10583wftddv3vqnn7cuqrsd84hj9r4u2yhjg5nd: validator does not exist: unknown request","time":"2022-09-09T15:29:40Z","message":"failed to get contract name"}

We have err point out this time the proposer is evmosvalcons10583wftddv3vqnn7cuqrsd84hj9r4u2yhjg5nd, that is "Smart Stake | evmos.smartstake.io" since block 873482 (not exists at request height = 59141)

And now this is log on my node:

3:29PM INF indexed block height=3775696 module=txindex server=node 3:29PM INF Served eth_call conn=my.ip:port duration=73.683148 err="rpc error: code = Unknown desc = rpc error: code = Internal desc = failed to obtain coinbase address: failed to retrieve validator from block proposer address evmosvalcons10583wftddv3vqnn7cuqrsd84hj9r4u2yhjg5nd: validator does not exist: unknown request" module=geth reqid=53 3:29PM INF Served eth_call conn=my.ip:port duration=1360.187138 err="rpc error: code = Unknown desc = rpc error: code = Internal desc = failed to obtain coinbase address: failed to retrieve validator from block proposer address evmosvalcons10583wftddv3vqnn7cuqrsd84hj9r4u2yhjg5nd: validator does not exist: unknown request" module=geth reqid=54 3:29PM INF Transactions per second tps=0.7 3:29PM INF executed block height=3775697 module=state num_invalid_txs=0 num_valid_txs=2 server=node

We see node indexed and say block 3775696 before the error message, and then node says INF executed block height=3775697 after the error message. You know what? evmosvalcons10583wftddv3vqnn7cuqrsd84hj9r4u2yhjg5nd aka Smart Stake | evmos.smartstake.io is the proposer of block 3775696

[VictorTrustyDev]

So @yihuang , you say occasionally fail is totally correct. When latest block on your node proposed by a validator who exists before context, it's ok. But if validator does not exists at context, it would fail

Days ago, I usually facing with this but it also occasionally and then alright later without knowing why so I just ignored it.

[github-actions[bot]]

This issue is stale because it has been open 45 days with no activity. Remove Status: Stale label or comment or this will be closed in 7 days.

[yihuang]

should be fixed by #1391