stacks-core [Atlas] Never give up trying to replicate attachments

We've recently had an issue reported in the Hiro API where a subdomain which was correctly registered as a name-update to BNS does not appear in our endpoint responses, redirecting the user to the Stacks registrar when querying for it.

Upon further investigation, I discovered that we did not receive the attachment message (/attachments/new RPC) that would've contained the updated zonefile with the new subdomain only in some of our API instances. In those cases, the API only registered the update transaction but never the subdomain.

Fortunately, the API instance that is currently live did receive that attachment and is displaying the name correctly, but we'd like to make sure we're covering all of our bases for processing attachments.

This brings me to my question:

Are attachments guaranteed to be transmitted by nodes? Or should we assume they are 'best effort' messages similar to mempool updates?

Thanks

Aug 31 '22 15:08 rafaelcr

Hey @rafaelcr,

Attachment replication is best-effort, because attachment data is not consensus-critical.

The node can be configured to track different contracts for attachment events, which are generated through (print ...) statements. By default, it tracks attachment events from .bns. Once these attachment events are logged, a separate subsystem -- the Atlas network (src/net/atlas) -- attempts to contact the peer's neighbors to see if any of them have the associated attachment data. Eventually, all nodes tracking attachments for the same contracts should receive the attachment. But, the node will only make ~50 attempts for an attachment until it gives up on the attachment altogether. So, periods of network unavailability or a dearth of neighbors with the attachment could prevent your node from receiving it.

The Atlas system could be made more reliable (e.g. the one in Stacks 1.0 never gave up, and we could make it so that this one never gives up either), but to answer your immediate question, the attachment replication system is very much best-effort, and there is no guarantee of replication like there is for blocks and microblocks.

Aug 31 '22 15:08 jcnelson

Thanks for the quick response @jcnelson 🤝 . That is very informative (cc @CharlieC3 )

Follow up: is there currently a way for us to request an attachment (or a zonefile) if we did not see it for some time after we receive a name-update?

Aug 31 '22 15:08 rafaelcr

Sure -- if you know the node that received it, you can query it with /v2/attachments/<attachment-20-byte-hash>

Aug 31 '22 16:08 jcnelson

Appreciate it @jcnelson , closing this as my question was answered

Aug 31 '22 21:08 rafaelcr

Going to reopen this. I think it's important to make sure that the Atlas state-machine always re-tries attachments (even if infrequently) in order to handle these kinds of corner cases.

A good acceptance test would be to see if we can instantiate a Stacks node, blow away its Atlas attachments, and verify that it's able to eventually re-download all of them.

Sep 01 '22 15:09 jcnelson

@jcnelson Thank you! Albeit not consensus-critical, missing attachments definitely impact the developer experience.

Sep 01 '22 15:09 CharlieC3

Hey all, as this is creating issues for a Sigle user, I was wondering if you have any idea of an ETA for this one?

Nov 01 '22 13:11 pradel

Sometime after Stacks 2.1 ships, at the earliest.

Nov 01 '22 13:11 jcnelson

Okay, thanks

Nov 01 '22 16:11 pradel

It seems that some zonefile where lost in the past few days. We had many Sigle users that had working blog and are now unable to access them as the API is returning an empty zonefile entry. Example: https://stacks-node-api.mainnet.stacks.co/v1/names/curator.btc

Mar 14 '23 14:03 pradel

More reports from Sigle related to this:

Hey all we have an increasing number of users at Sigle that seems to have issues with the API and zonefile attachements, they go to btc.us add the Gaia url in the input to link it to their name submit the tx and the zonefile field is still empty. Some affected users:

https://stacks-node-api.mainnet.stacks.co/v1/names/sam-tlt.btc

https://stacks-node-api.mainnet.stacks.co/v1/names/mendelmire.btc Some people managed to get it work but just resubmitting the same transaction again and again but for some that's not working

I know @lgalabru and @jbencin have been working on improving Atlas. Can this issue be moved out of the icebox and into development? For example these PRs are at least related:

https://github.com/stacks-network/stacks-blockchain/pull/3618
https://github.com/stacks-network/stacks-blockchain/pull/3612

Mar 21 '23 15:03 zone117x

I've been looking over the Atlas code and I think this behavoir is controlled by unresolved_attachment_instances_expire_after in AtlasConfig. If you use the code from PR #3618, you can set this value in Stacks.toml up to a limit of u32::MAX, which is about 136 years. This should work as a temporary fix, at least for the next 135 years or so.

A better solution might be to attempt to download an attachment frequently at first, and retry with exponentially increasing delay after each failure, up to some maximum limit (like 24 hours), but never stop trying.

@CharlieC3 and @zone117x, can you try the proposed workaround and let me know if it works?

Mar 23 '23 21:03 jbencin

Hey all, just wanted to see if any progress has been made on that one as this is still affecting users on Sigle side.

Aug 22 '23 08:08 pradel

@pradel See my comment above. PR #3618 has added config options that should fix this, and has been approved and merged into develop, but has not yet made it into master or an official release. If you want to fix this now, you could try building it yourself by checking out the 2.4.0.0.0 tag and manually changing some default values:

Clone the project

git clone [email protected]:stacks-network/stacks-blockchain.git
cd stacks-blockchain
git checkout 2.4.0.0.0

Edit the default config

Open src/net/atlas/mod.rs and update the values in AtlasConfig to something like this:

        AtlasConfig {
            contracts,
            attachments_max_size: 1_048_576,
            max_uninstantiated_attachments: 10_000,
            uninstantiated_attachments_expire_after: 3_600,
            unresolved_attachment_instances_expire_after: u32::MAX,
            genesis_attachments: None,
        }

unresolved_attachment_instances_expire_after controls how long to look for attachments which have a corresponding hash on the blockchain. By setting it to the max value, we'll continue looking for over 100 years.

Build the `stacks-node` binary

cd testnet/stacks-node
cargo build --features monitoring_prom,slog_json --release

Aug 29 '23 15:08 jbencin

Is there any reason why users are currently experiencing this issue much more frequently? (not just related to subdomains but regular names too)

Bilal from btc.us found that out of the 28 successful transactions (between the 15th and 20th of Feb.) name-update only 6 show zone files on the API. Users use to have a chance at getting an update to stick but now they can send dozens without any change.

Example Transaction 20 days ago to add lightning address to zone file was fine and worked. Subsequent transactions had no effect https://explorer.hiro.so/address/SP3WE8GKTG0MVP2NRV8C7CM3ZFFB3QRXZXMWHBJ1F?chain=mainnet

https://stacks-node-api.mainnet.stacks.co/v1/names/franek.btc/zonefile results in { "zonefile": "" }

Another one like it here https://stacks-node-api.mainnet.stacks.co/v1/names/cryptostonks.btc/zonefile several attempts from the user to get a zonefile added.

@jcnelson can this fix be included in the next version?

Feb 23 '24 11:02 314159265359879

It doesn't get picked up by the api, 4 attempts in 2 days: https://api.mainnet.hiro.so/v1/names/nebo.btc/zonefile { "zonefile": "" }

name-register https://explorer.hiro.so/txid/0x274fdc78ac8210ee4a2488f0fbfe181fd2522096f81d70d5c29ae17d38863bb8?chain=mainnet transaction 0 name-update https://explorer.hiro.so/txid/0x4f1594fe0a842361f3025335cf42308b5771f75e403cb26e00a3e74a26fed4a4?chain=mainnet transaction 1 name-update https://explorer.hiro.so/txid/0x1a6f2058d40bad45ad4219ea6c837b306bd63e474c1738361ea988abcd0c8ca0?chain=mainnet transaction 2 name-update https://explorer.hiro.so/txid/0x470d21789e221164cc3eade7b114ab4a36e4ece1b383c6417fcb7bad69cb77b7?chain=mainnet

Mar 11 '24 20:03 314159265359879

i've been trying for days to update my domain at btc.us and the changes are never reflected. specifically, i'm trying to add a bitcoin address and redirect url https://explorer.hiro.so/txid/0x71e9083c8dbe35b9793a1fc62108a62fbe9e7c06143da74aee53d2433d1ad20b?chain=mainnet https://explorer.hiro.so/txid/0xef000472da40219e7d3e23631db1ab1aa2a4ec3d60b39c07e054920e9724c236?chain=mainnet

i've tried on my other domain names to do same update (add btc address and redirect url) and i don't know what i'm doing wrong. please help. thank you.

Mar 22 '24 06:03 nin-j

domain error

https://explorer.hiro.so/txid/0x0b90a7afaf5107dd9fcfd72473b8d30f9e5c9eee144665536a4ec982827ed05b?chain=mainnet https://explorer.hiro.so/txid/0x68c96edf6a800f9a579eae507812ce0b699fd3aeb1cfd032a41c5bbaccf7806d?chain=mainnet

Mar 24 '24 07:03 slofslb

Hi, I experienced an error as well.

Tried with 2 txns to update my lightning wallet and btc wallet connected to my .btc address (colinpower.btc), but it did not work either time.

Appreciate your help looking into this! I'm very excited to use my .btc address.

Mar 27 '24 05:03 colinpower

per tx 0xdccb0d81eb790cc07698ba2fcfe288cdf0dd0888e5febbf1a8f16e2bdc75dc8b via btc.us I called name-update to add a profile uri record for citycoinstacker.btc to add https://gaia.hiro.so/hub/1Q9PqNXToxSjZnHayN6oUxmdUKY5w1xWhe/profile.json

however after that tx confirmed, I get https://api.mainnet.hiro.so/v1/names/citycoinstacker.btc/zonefile and it returns:

{
  "zonefile": ""
}

As requested by the banner on btc.us, I'm adding my situation comment.

Mar 27 '24 23:03 cryptocracy

per tx 0xdccb0d81eb790cc07698ba2fcfe288cdf0dd0888e5febbf1a8f16e2bdc75dc8b via btc.us I called name-update to add a profile uri record for citycoinstacker.btc to add https://gaia.hiro.so/hub/1Q9PqNXToxSjZnHayN6oUxmdUKY5w1xWhe/profile.json

however after that tx confirmed, I get https://api.mainnet.hiro.so/v1/names/citycoinstacker.btc/zonefile and it returns:
{
  "zonefile": ""
}
As requested by the banner on btc.us, I'm adding my situation comment.

is there any solution to this? I am facing similar issue

Apr 04 '24 11:04 sunnykinger

Some clarity on the future of BNS would be helpful. Does BNS have a Product Owner? If not, why not? Who is driving development? How is orange domains going to fit in? Where's the roadmap? I get that there are other priorities, but having no communication on the roadmap and ongoing open issues is really detrimental to a potential key part of the ecosystem.

Apr 14 '24 18:04 dantrevino

Hey! Appreciate the flagging @dantrevino - all on the same team with the same desire to see BNS have more functionality. OD is absolutely a stakeholder in BNS and an update will be shared shortly in terms of the ongoing efforts there; nothing is happening without community participation. I'd also love to learn more about what you're building on BNS (and others on this thread too), would be great to have a public call or spaces where we can all meet soon; I'll take that as an action item on my end over the next few weeks! 🙂

Apr 18 '24 17:04 GinaAbrams

@314159265359879 do we know which Stacks node btc.us is using to broadcast its BNS transactions?

I'm in the process of getting familiar with the Atlas code to understand how the attachment flow works and I suspect there are other issues that could be getting in the way of attachment replication. For example, I believe AtlasConfig values for max_uninstantiated_attachments (defaults to 50,000) and uninstantiated_attachments_expire_after (defaults to 1 day) should also be set to higher values in the node that is receiving the name registration transactions (cc @jbencin ). There are no debug logs printed when attachments are evicted based on these configs so right now it's also difficult to know if this is the underlying issue.

Could they try this on the relevant Stacks node to see if UX improves? If they use Hiro nodes I can coordinate this change on our end ASAP to see if that works.

Apr 27 '24 23:04 rafaelcr

@rafaelcr I will forward this to someone who knows more.

Apr 27 '24 23:04 314159265359879

@rafaelcr btc.us uses @stacks/connect to broadcast all its transactions, so I'm not exactly sure what stacks node is in use.

Additionally we do not provide a coreNode property to our UserSession object in @stacks/connect (prompting a fallback to the default value). Which seems to be 'https://api.mainnet.hiro.so' according to this and this

Happy to coordinate on any experiments that need to be run.

Apr 29 '24 08:04 bilalanees98

@Gina Abrams @.***> my user case is related to wallet functionality. Ultimately I'm interested in how/where profile data will be found.

Typically profile and bns are two different things, but we're setting some metadata related to nostr and lightning in BTC.us.

So I would like to understand how/if profiles will be managed in the future? StackerDB? Gaia? Somewhere else? Are all update functions in the @stacks/profile lib? Or are updates required?

Dan

On Thu, Apr 18, 2024, 12:03 PM Gina Abrams @.***> wrote:

Hey! Appreciate the flagging @dantrevino https://github.com/dantrevino

all on the same team with the same desire to see BNS have more functionality. OD is absolutely a stakeholder in BNS and an update will be shared shortly in terms of the ongoing efforts there; nothing is happening without community participation. I'd also love to learn more about what you're building on BNS (and others on this thread too), would be great to have a public call or spaces where we can all meet soon; I'll take that as an action item on my end over the next few weeks! 🙂

— Reply to this email directly, view it on GitHub https://github.com/stacks-network/stacks-core/issues/3270#issuecomment-2064577704, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAB4OUBQL2MSZ3NVXZ6BIDY574HFAVCNFSM6AAAAAAQBPE57GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRUGU3TONZQGQ . You are receiving this because you were mentioned.Message ID: @.***>

May 07 '24 01:05 dantrevino

Hey! Appreciate the flagging @dantrevino - all on the same team with the same desire to see BNS have more functionality. OD is absolutely a stakeholder in BNS and an update will be shared shortly in terms of the ongoing efforts there; nothing is happening without community participation. I'd also love to learn more about what you're building on BNS (and others on this thread too), would be great to have a public call or spaces where we can all meet soon; I'll take that as an action item on my end over the next few weeks! 🙂

At Paradigma we have launched https://ubid.app that empowers web applications with BNS names. It let's you use your BNS name as user name, and defining a password to login to a web2 app. It uses the OpenID Connect protocol defined as base standard for web apps in many countries and continents of the world for interoperability. The defined password never leaves the user browser making it one safest's solution on the world!