Invalid milestone timeout and strong log severities

[sammy007]

v1.0.5

Heimdall seems running seamless, Bor is in sync and no issues as far as I can tell, suddenly getting many ERRORs which if my assumption is correct are not really errors but just WARNINGs.

ERROR[2024-04-18|01:36:54.716] Older invalid tx found                       module=staking
ERROR[2024-04-18|01:37:01.175] Older invalid tx found                       module=staking
ERROR[2024-04-18|01:37:27.245] Error validating milestone
ERROR[2024-04-18|01:37:58.755] Invalid Milestone Timeout msg
ERROR[2024-04-18|01:38:11.165] Hash is not valid                            module=Milestone
ERROR[2024-04-18|01:38:23.854] Invalid Milestone Timeout msg                module=Milestone lastMilestoneTime=2024-04-18T01:38:17Z currenttime=2024-04-18T01:38:17.114447618Z bufferTime=4m16s
ERROR[2024-04-18|01:38:23.856] Milestone not in continuity                  module=Milestone lastMilestoneEndBlock=55949103 receivedMsgStartBlock=55949043
ERROR[2024-04-18|01:38:23.856] Invalid Milestone Timeout msg                module=Milestone lastMilestoneTime=2024-04-18T01:38:17Z currenttime=2024-04-18T01:38:17.114447618Z bufferTime=4m16s
ERROR[2024-04-18|01:38:23.857] Invalid Milestone Timeout msg                module=Milestone lastMilestoneTime=2024-04-18T01:38:17Z currenttime=2024-04-18T01:38:17.114447618Z bufferTime=4m16s
ERROR[2024-04-18|01:38:23.857] Invalid Milestone Timeout msg                module=Milestone lastMilestoneTime=2024-04-18T01:38:17Z currenttime=2024-04-18T01:38:17.114447618Z bufferTime=4m16s

In case it's normal and does not cause serious issues why at first place they are ERRORs? Such strong severity without a strong particular reason leads to a lot of false positive alerts in case automated log analysis is used, it's basically triggers anomaly alerts.

Could you please consider changing severities or is it signaling that node is sick?

I can tell the same about Bor which does the opposite (borrowed from Bor issues):

WARN [11-30|11:10:27.352] Synchronisation failed, dropping peer    peer=1673b28af0061121f1ec941aae813b2c5907d3b7fcde0746aee66d28ce2e8247 err="retrieved hash chain is invalid: invalid merkle root (remote: 55f9a3e801a05d3330f06aa1a9ba9a09fe7979230d50846bb9d886c96383755e local: e18a8fbe060ebefdc86cd2f2a5d0c14f70f6dfcc7252efe232c4c867ef37a01f) dberr: %!w(<nil>)"
WARN [11-30|11:10:27.352] Synchronisation failed, retrying         peer=1673b28af0061121f1ec941aae813b2c5907d3b7fcde0746aee66d28ce2e8247 err="peer is unknown or unhealthy"
INFO [11-30|11:10:34.342] Got new milestone from heimdall          start=50,570,307 end=50,570,319 hash=0xf1d3d69bd5d203c84d9e697a47ef078b91bea1b230508f3e6013aefde6a6aa1f
WARN [11-30|11:10:34.343] unable to handle whitelist milestone     err="missing blocks"
INFO [11-30|11:10:46.343] Got new milestone from heimdall          start=50,570,307 end=50,570,319 hash=0xf1d3d69bd5d203c84d9e697a47ef078b91bea1b230508f3e6013aefde6a6aa1f
WARN [11-30|11:10:46.343] unable to handle whitelist milestone     err="missing blocks"
INFO [11-30|11:10:58.343] Got new milestone from heimdall          start=50,570,307 end=50,570,319 hash=0xf1d3d69bd5d203c84d9e697a47ef078b91bea1b230508f3e6013aefde6a6aa1f
WARN [11-30|11:10:58.343] unable to handle whitelist milestone     err="missing blocks"

If synchronisation fails it's a serious issue and its severity must be ERROR.