constant error messages on explorer node

[LeoHChen]

Describe the bug A clear and concise description of what the bug is.

I've seen those error log constantly on explorer nodes. We need to do some deep dive on them.

{"level":"info","port":"9000","ip":"54.189.61.183","error":"leveldb: not found","caller":"/home/lc/go/src/github.com/harmony-one/harmony/core/rawdb/accessors_offchain.go:143","time":"2020-07-25T08:47:50.71223974Z","message":"ReadValidatorSnapshot"}
{"level":"info","port":"9000","ip":"54.189.61.183","error":"leveldb: not found","caller":"/home/lc/go/src/github.com/harmony-one/harmony/core/rawdb/accessors_offchain.go:143","time":"2020-07-25T08:47:50.713232162Z","message":"ReadValidatorSnapshot"}
{"level":"info","port":"9000","ip":"54.189.61.183","error":"leveldb: not found","caller":"/home/lc/go/src/github.com/harmony-one/harmony/core/rawdb/accessors_offchain.go:143","time":"2020-07-25T08:47:50.720177732Z","message":"ReadValidatorSnapshot"}
{"level":"info","port":"9000","ip":"54.189.61.183","error":"leveldb: not found","caller":"/home/lc/go/src/github.com/harmony-one/harmony/core/rawdb/accessors_offchain.go:143","time":"2020-07-25T08:47:50.725154385Z","message":"ReadValidatorSnapshot"}
{"level":"warn","port":"9000","ip":"54.189.61.183","error":"min_self_delegation 10000000000000000000000, amount 0: self delegation can not be less than min_self_delegation","hash":"0xe3844b0193414ed2fb09159361ba4c5b04420a249cb1529f0007a15a4ca36b2d","caller":"/home/lc/go/src/github.com/harmony-one/harmony/core/tx_pool.go:897","time":"2020-07-25T08:47:54.679354147Z","message":"Discarding invalid transaction"}
{"level":"info","port":"9000","ip":"54.189.61.183","length of newStakingTxs":1,"totalPending":0,"totalQueued":0,"caller":"/home/lc/go/src/github.com/harmony-one/harmony/node/node.go:233","time":"2020-07-25T08:47:54.679405707Z","message":"Got more staking transactions"}
{"level":"info","port":"9000","ip":"54.189.61.183","error":"self delegation can not be less than min_self_delegation","caller":"/home/lc/go/src/github.com/harmony-one/harmony/node/node.go:248","time":"2020-07-25T08:47:54.679416167Z","message":"[AddPendingStakingTransaction] Failed adding new staking transaction"}
{"level":"info","port":"9000","ip":"54.189.61.183","Hash":"0xe3844b0193414ed2fb09159361ba4c5b04420a249cb1529f0007a15a4ca36b2d","caller":"/home/lc/go/src/github.com/harmony-one/harmony/node/node.go:254","time":"2020-07-25T08:47:54.679424137Z","message":"Broadcasting Staking Tx"}
{"level":"info","port":"9000","ip":"54.189.61.183","shardGroupID":"harmony/0.0.1/node/beacon","caller":"/home/lc/go/src/github.com/harmony-one/harmony/node/node.go:189","time":"2020-07-25T08:47:54.679447067Z","message":"tryBroadcastStaking"}
{"level":"warn","port":"9000","ip":"54.189.61.183","error":"self delegation can not be less than min_self_delegation","caller":"/home/lc/go/src/github.com/harmony-one/harmony/internal/hmyapi/apiv1/util.go:68","time":"2020-07-25T08:47:54.682426016Z","message":"Could not submit staking transaction"}

[rhazberries]

I think the transaction errors are fine. They just mean that someone sent an invalid staking transaction the the chain.

[LeoHChen]

The transaction errors are never gone. Which is weird though.

[JackyWYX]

Have we directly copied db from archive validator node to explorer? DB not found might also be caused by some data is not written in validator node but in explorer node. api/service/explorer/storage.go:27