When re-creating listeners in EVMConnect/EthConnect/FabConnect use the last event block

[peterbroadhurst]

See #1531

Every time Core starts, it confirms all the listeners exist that it needs for all of:

• Contract listeners registered by applications
• BatchPin listeners for any multi-party namespaces
• Token listeners - indirectly via /activatepool
  • Not changed in this PR - see #1531 comments for justification

This is needed for two cases:

1. In case the listeners were deleted operationally for any reason (intentionally or otherwise) - such as a DR scenario
2. In the v1.2->v1.3 migration, where we intentionally orphan all the core-wide listeners in favor of namespace-specific ones

In both of these cases, the logic was naive for the newest/latest case - where it treats "now" as the latest event.

That is a problem, as it might be a long time since the original listener was created, and it might have delivered many events already. In which case there might be "gaps" we skip over between the last event detected by the old listener, and the first event we detect on the new one.

This PR closes that window, by looking at the protocolId of the last event - which is architecturally required to be a lexicographically sorted string identifying the position of the event on the blockchain.

If there's been a previous event, that's used by the blockchain connector as the fromBlock for the new listener.

Also, if "oldest" or an explicit block number is used, and the block number is before the block determined by the above process, then the newer block is used.

[peterbroadhurst]

PR has been updated based on the conversation here, so that in the case a specific block number is supplied the recreated listener will still restart on the block before the last event (if that's after the specified block): https://github.com/hyperledger/firefly/issues/1531#issuecomment-2187054888

[EnriqueL8]

Will be testing this one today e2e

[EnriqueL8]

Ah you covered this already https://github.com/hyperledger/firefly/pull/1534/files#diff-eb673d841796eecd85e0b4f676fb8de03a03b31b2a7d66a5c63d600aab57e2dfR217
 sorry

[EnriqueL8]

Have tested this one E2E by doing the following experiment:

• Setup a 1.3.0 FF stack
• Deploy a contract and a listener with the option to listen from newest
• Delete the listener from EVMConnect
• Restart the FF with the code from this PR
• Find the new listener created in EVMConnect and make sure the fromBlock matches the last protocol ID for the last event - 1

[peterbroadhurst]

we recreate a listener with block number of the last protocol ID but we haven't consumed all the events from that block could we lose those events? Is it safer to go one down of the procotol id or am I over complicating this?

This is the reason we go one block back: https://github.com/hyperledger/firefly/blob/8dd5a0d2899a0b002d080a60f15ab64770ef1be0/internal/blockchain/ethereum/eventstream.go#L216-L218

[EnriqueL8]

Yeah I realised with https://github.com/hyperledger/firefly/pull/1534#issuecomment-2205567325