zeta-chain
Missed solana outbound on mainnet
Describe the Bug
On mainnet, this outbound QtCixwj7shQab8iUR1rsiKSHcTajToZZqJZexrdk73DkMULt3UgXV4EZxnufKYCCUQMHrL2Qjyxmw8gNTzmzRe2 was completely missed and no logs related to it are visible in zetaclients.
It was relayed by external relayer, and it's currently not clear if some error happened after broadcast.
Purpose of this issue is to investigate if something can happen after broadcast that is relayer-specific, for example reportToOutboundTracker and if reportToOutboundTracker can be improved so it is not done only by relayer. Otherwise, errors or anything else that can cause this happening on relayer only are not visible, and in general reporting to outbound tracker is depending on relayer only, and successful outbounds can be missed.
NOTE: This was solved by identifying transaction manually and adding outbound tracker.
The idea solution is to switching tracker report model to signature scanning model, like inbound transaction. But another way to avoid missing outbound tracker is to implement a similar to EVM chain tracker supplement, and the implementation is cheap and fast for Solana. Not sure what's the priority set for this issue but I can always help.
Can it be closed now? Removing the bug label if we just need to add more logs
Can it be closed now? Removing the bug label if we just need to add more logs
We think we hit it again today on athens so no: https://explorer.solana.com/tx/5VFvzhr8BHkcxvYzcmNnQjMpu82C2ScKxZNjB9RbQp3iUSj8UwSfB9bhHexL9gCS9ZapFsP5opdDfFwTZy2r7raC?cluster=devnet
error GetTransaction for untilSig 5VFvzhr8BHkcxvYzcmNnQjMpu82C2ScKxZNjB9RbQp3iUSj8UwSfB9bhHexL9gCS9ZapFsP5opdDfFwTZy2r7raC: rpc call getTransaction() on https://solana-devnet.g.allthatnode.com/full/json_rpc/978814a9d9f0499984805e52e3db081a: Post "https://solana-devnet.g.allthatnode.com/full/json_rpc/978814a9d9f0499984805e52e3db081a": context canceled
agree, seems like medium-high priority issue, as its enough that 1 relayer has some error or down time for tx to be missed, we need something similar to evm model so reporting to outbound tracker is not coupled to relayer that broadcasted tx
It makes sense to have a supplemental goroutine to check for missed outbounds in the zetaclient. It should be easy enough to add and solve the problem to some extent
Additionally, eventual support of solana for the outbound tracker creation tool would also help. Tracked here (https://github.com/zeta-chain/infrastructure/issues/2284)
Closing as we now have more reliable infrastructure to report missed outbound, in particular from https://github.com/zeta-chain/infrastructure/issues/2284