Mandatory upgrade version v11.0.0 is tagged as "pre-release"

[08d2]

My full node halted at height 5432450 claiming UPGRADE v11 NEEDED. But v11.0.0 is currently tagged as a pre-release. How are node operators expected to discover and apply required upgrades like this one?

[czarcas7ic]

Hey! The reason why we kept it as pre-release is to prevent people from trying to run v11 prior to the upgrade height. But to answer your question, all future upgrade announcements will be under the new "mainnet-anouncements" channel in our discord.

[08d2]

prevent people from trying to run v11 prior to the upgrade height

Node operators must deploy and run v11 prior to the upgrade height. If they don't, their nodes will halt at that height. Right? Am I mistaken?

all future upgrade announcements will be under the new "mainnet-anouncements" channel in our discord.

If you announce upgrade requirements exclusively via Discord, how can I reliably run an Osmosis full node? It's obviously infeasible for me to join a chat room for each network I am participating in, read the chat logs on an O(daily) cadence, and manually perform whatever operational work that's (informally) implied by the chat messages I see.

Respectfully, I'd request that the issue be re-opened, because I don't believe my concern has been addressed.

[czarcas7ic]

Node operators must deploy and run v11 prior to the upgrade height. If they don't, their nodes will halt at that height. Right? Am I mistaken?

Somewhat, the binary can be prepped within something like cosmovisor, but many are mistaken and try to run the next binary before the upgrade height and will get an app hash mismatch.

If you announce upgrade requirements exclusively via Discord, how can I reliably run an Osmosis full node? It's obviously infeasible for me to join a chat room for each network I am participating in, read the chat logs on an O(daily) cadence, and manually perform whatever operational work that's (informally) implied by the chat messages I see.

IMO Discord is the best solution to this, since you can subscribe to a single channel (in this case the mainnet announcements channel) and would only get pinged when an upgrade is happening. The next best thing would be to look out on the proposals page but this would be more manual than the prior suggestion.

[ValarDragon]

So theres two separate issues here:

• v11 release being marked as pre-release
• What is the intended notification flow for full nodes

And theres a path for automatically upgrading your binaries if you don't want to follow along the notifications to upgrade You can configure cosmovisor to allow automatically downloading of the next release binary.

pre-release

I actually agree that v11 shouldn't have been marked as pre-release for this long. We probably should have a system for marking it as a true release once some amount of yes votes are on, and make it clear in the release notes + errors on mainnet if your running the wrong version

intended notification flow for full nodes

Theres currently two notification flows:

• The blockchain governance and explorers - they show the governance proposal when an upgrade is happening, and when its approved. Granted this is not a great system
• The Osmosis validators discord (Granted there should be something more permissive just for full node operators as well)
  • Its possible to subscribe to posts in that announcements channel to have them appear on any personal discord. I think theres some webhook hacks to make it possible in slack

Your absolutely right, and this has been under-explored, that there should be more accessible notification flows. Following the github releases was not an intended one, but I think it makes sense to support.

Some ideas:

• An official updates twitter?
• An official updates email? (A bit complex, this email should probably only get emailed once a software upgrade proposal hits 66+% yes? Not sure how this gets maintained? Ideally we could make something on explorers that you can subscribe to notifications

Any other ideas for how notifications for full nodes for upgrades can happen? Is github releases sufficient?

[tsenart]

+1 for Github releases, which you can subscribe to and get notifications for. My expectation is that every node operator would have at least 1 week of notice before the cut-off height.

The Osmosis validators discord

Is this a different Discord than the Osmosis discord server? I'd be happy to join if it's open, but heavily prefer the GitHub releases signal to noise ratio, assuming you take care of them going forward.

[08d2]

prefer the GitHub releases signal to noise ratio

+1. I can't realistically join and monitor a Discord for each network where I operate nodes.

I can't use cosmovisor, and need at least a few days notice to prepare for breaking version upgrades.