[Atlas] Never give up trying to replicate attachments

[rafaelcr]

We've recently had an issue reported in the Hiro API where a subdomain which was correctly registered as a name-update to BNS does not appear in our endpoint responses, redirecting the user to the Stacks registrar when querying for it.

Upon further investigation, I discovered that we did not receive the attachment message (/attachments/new RPC) that would've contained the updated zonefile with the new subdomain only in some of our API instances. In those cases, the API only registered the update transaction but never the subdomain.

Fortunately, the API instance that is currently live did receive that attachment and is displaying the name correctly, but we'd like to make sure we're covering all of our bases for processing attachments.

This brings me to my question:

Are attachments guaranteed to be transmitted by nodes? Or should we assume they are 'best effort' messages similar to mempool updates?

Thanks

[jcnelson]

Hey @rafaelcr,

Attachment replication is best-effort, because attachment data is not consensus-critical.

The node can be configured to track different contracts for attachment events, which are generated through (print ...) statements. By default, it tracks attachment events from .bns. Once these attachment events are logged, a separate subsystem -- the Atlas network (src/net/atlas) -- attempts to contact the peer's neighbors to see if any of them have the associated attachment data. Eventually, all nodes tracking attachments for the same contracts should receive the attachment. But, the node will only make ~50 attempts for an attachment until it gives up on the attachment altogether. So, periods of network unavailability or a dearth of neighbors with the attachment could prevent your node from receiving it.

The Atlas system could be made more reliable (e.g. the one in Stacks 1.0 never gave up, and we could make it so that this one never gives up either), but to answer your immediate question, the attachment replication system is very much best-effort, and there is no guarantee of replication like there is for blocks and microblocks.

[rafaelcr]

Thanks for the quick response @jcnelson 🤝 . That is very informative (cc @CharlieC3 )

Follow up: is there currently a way for us to request an attachment (or a zonefile) if we did not see it for some time after we receive a name-update?

[jcnelson]

Sure -- if you know the node that received it, you can query it with /v2/attachments/<attachment-20-byte-hash>

[rafaelcr]

Appreciate it @jcnelson , closing this as my question was answered

[jcnelson]

Going to reopen this. I think it's important to make sure that the Atlas state-machine always re-tries attachments (even if infrequently) in order to handle these kinds of corner cases.

A good acceptance test would be to see if we can instantiate a Stacks node, blow away its Atlas attachments, and verify that it's able to eventually re-download all of them.

[CharlieC3]

@jcnelson Thank you! Albeit not consensus-critical, missing attachments definitely impact the developer experience.

[pradel]

Hey all, as this is creating issues for a Sigle user, I was wondering if you have any idea of an ETA for this one?

[jcnelson]

Sometime after Stacks 2.1 ships, at the earliest.

[pradel]

Okay, thanks