Do we really need fleet state checksum?

[fjarri]

Initially it was introduced as a performance optimization to prevent sending the metadata to another node if both node have matching fleet states. But how often does that really happen in reality? From my observations on the mainnet, my node changed fleet states rather frequently. Every time Ursula is restarted, its metadata changes, changing the fleet state checksum. Since the mainnet is currently down, it's hard to verify - need to wait.

A possible way to handle this would be to ensure that each node's metadata is as stable as possible:

• remove the timestamp from it, or not include it when calculating the checksum
• do not include the metadata signature in the checksum (it is randomized)
• ensure that the certificate stays the same when node is restarted (probably does, but need to check)

Or, we can get rid of the fleet state concept completely. This will allow us to send much less information about known nodes - basically just hosts, ports, and staking provider addresses (to know which addresses to query on retrieval, for example), possibly saving traffic as a result. We need to get the node's metadata from it personally anyway to finalize verification, so there's no need to get it from someone else.

[fjarri]

Hm, or should this have been filed in nucypher-core repo?

[cygnusv]

I think this issue is also related to https://github.com/nucypher/nucypher/issues/1000

[jMyles]

I agree that the FleetState checksum needs to reflect (which is to say, match) the output of successful comprehension of the network.

So the questions are: is comprehension of the entire network a cognizable concept? and is comprehension a plausible basis for significant optimization?

I continue to think that, for small networks (meaning less than about 5000 or so nodes), the answer to the former is "yes". And I think that the answer to the second is "yes."

Obviously an timestamp reflecting node uptime is an absurd object for the checksum. It was never meant to be that; it was meant to reflect material generation time, not uptime. Omitting it is perhaps plausible, but if this is done, how will learners know that metadata is updated? Will it need to be versioned or something?

[fjarri]

The question is, is it really necessary to have a "comprehension of the network"? What is the use of knowing that there is some metadata out there if your node hasn't personally verified it?

Omitting it is perhaps plausible, but if this is done, how will learners know that metadata is updated? Will it need to be versioned or something?

In my implementation I avoid this question by simply ignoring all the metadata except for the one the node got by querying /public_information and verifying it. All the metadata I get from learning is reduced to (host, port, staker_address), and the metadata I get from external POST requests is further filtered to only those items where host == request.remote_addr. Additionally, if I notice during learning that the teacher supplied a different metadata for itself compared to what I have on record, I invalidate its verified status and mark it for re-verification. This way I don't need to care which metadata is more recent - whatever the node reports about itself is the single source of truth.

P.S. I keep the staker_address mostly for Alice/Bob's benefit (so that they know whom to contact to possibly reach a specific staker), Ursulas don't really need that either.