joystream icon indicating copy to clipboard operation
joystream copied to clipboard

Carthage rollout plan

Open bedeho opened this issue 2 years ago • 8 comments

Background

We are launching Carthage, which will be a new chain final testnet who's launch path should be as similar as possible as mainnet.

Proposal

The rollout starts with the following genesis block and progresses the following three stages, each corresponding to a separate runtime

Stage (Runtime) Duration Sudo Validators User Action Space Jsgenesis Actions Staking Rewarded max_validator_count
1 Frozen Hours Yes (Single Key) PoA Staking & Nomination & Multisig Bootstrapping, Validation, call force_new_era using Sudo to go to Thawn No 12
2 Thawn Days Yes (Multisig) PoS Staking & Nomination & Multisig & Validation Validation, Increase validator count using Sudo, finally runtime upgrade to Supervised Yes 12-16
3 Supervised Days/Weeks Yes (Multisig) PoS Everything* Increase validator count using Sudo, and set Sudo to NULL Yes 12-24
4 Liberated Unlimited No PoS Everything* None Yes <council decides>

*Everything: this is not literally everything, its all non-frozen features (add list?)

Note that initial validators in Frozen will loose their slots from Thawn on by third party staking on new validators.

When we have finalized parameters, we should explicitly state

  • Howe elections start, whether they will fail, etc.
  • Same with actual validation?
  • anything else that depends on parameters?

┆Issue is synchronized with this Asana task by Unito

bedeho avatar May 25 '22 17:05 bedeho

Test Plan Sketch

Frozen

Martin does it all:

  1. Start Network
  2. Verify that block production and finalization is working.
  3. Verify balances.
  4. Verify that membership creation and balance transfers does not work with account X.
  5. Verify that nomination and validation candidacy announcement works, even using vesting funds!
  6. Verify that no staking rewards or era points are being earned.
  7. Verify that validators do receive tips, but do not receive any transaction fee cut.
  8. Verify Sudo works?
  9. Run Bootstrapping script.
  10. Verify that memberships were created properly.
  11. Sudo sets new sudo to Jsgenesis Sudo multisig account. (Make sure multisig account is funded before hand)
  12. Run force_new_era as Sudo.

Thawn

  1. Martin: Verify that validating with new actor is working: ... what account? [add more detail Martin]
  2. Martin: Verify that era points and rewards are earned and paid out?
  3. Martin & Bedeho & Mokhtar (Sudo multisig): Add 1 more validator slot
  4. Martin: Verify that membership creation and balance transfers does not work with account X.
  5. Martin: Verify that council election are progressing as they should.
  6. Martin: Verify that validators do receive tips, but do not receive any transaction fee cut.
  7. Martin & Bedeho & Mokhtar (Sudo multisig): Perform runtime upgrade to go to Supervised

Supervised

  1. Martin: Verify that all features work: membership creation, announcing, content directory.... [specify further].
  2. Martin & Bedeho & Mokhtar (Sudo multisig): Add 1 more validator slot
  3. Martin: Verify that validating with new actor is working: ... what account? [add more detail Martin]
  4. Martin: Verify that era points and rewards are earned and paid out?
  5. Martin: Verify that council election are progressing as they should.
  6. Set invulnerable validators to empty [] with sudo.
  7. Martin & Bedeho & Mokhtar (Sudo multisig): Set sudo to null , to go to Liberated

Liberated

  • no sudo possible with prior account or any other random account.
  • wip: verify control over all Jsgenesis accounts.

bedeho avatar Sep 27 '22 10:09 bedeho

Questions:

  • Will this test use the same parameters as for carthage?

    • that would mean we need a 9 validator nodes and a LOT of patience for testing last two stages especially.
  • Last two points on Supervised seems off. I assume the Council will not do the runtime upgrade, so they should be reversed.

  • I assume this is stated somewhere, but how many authorities can go down in Frozen without stopping finalization/production? I think testing that, and again later with other nodes, makes sense.


Added a lot of context to the plan, but I'll hold on adding them before we have clarified the above.

bwhm avatar Oct 09 '22 15:10 bwhm

Will this test use the same parameters as for carthage?

Yes indeed, not sure what you are referring to in terms of the last two stages, but any deviation should be avoided if possible.

Last two points on Supervised seems off. I assume the Council will not do the runtime upgrade, so they should be reversed.

These are problematic?

Set invulnerable validators to empty [] with sudo.
Martin & Bedeho & Mokhtar (Sudo multisig): Set sudo to null , to go to Liberated

what is wrong?

I assume this is stated somewhere, but how many authorities can go down in Frozen without stopping finalization/production? I think testing that, and again later with other nodes, makes sense.

That would just be to test GRANDPA itself, I don't think that makes sense?

Added a lot of context to the plan, but I'll hold on adding them before we have clarified the above.

I don't know what you are referring to here, what did you add, where?

bedeho avatar Oct 09 '22 17:10 bedeho

Yes indeed, not sure what you are referring to in terms of the last two stages, but any deviation should be avoided if possible.

I mean testing that all the transactions and flows works as intended. It's going to take 9+3+3 days just to get a council elected, unless we do it by sudo. Without a council, we can't hire leads, which means we can't test anything really.

Martin & Bedeho & Mokhtar (Sudo multisig): Set sudo to null , to go to Liberated

what is wrong?

I read it as first set sudo to null, then upgrade to liberated with the sudo key.

That would just be to test GRANDPA itself, I don't think that makes sense?

Probably not, as long as we're aware of how many we can lose.

I don't know what you are referring to here, what did you add, where?

I should say I wrote them, but didn't add them before we have clarified the above. Of which the big issue remaining is how to properly test functionality without changing some parameters. Without looking, I suspect proposal grace as well, but at the very least the election cycle.

bwhm avatar Oct 09 '22 17:10 bwhm

  • I assume this is stated somewhere, but how many authorities can go down in Frozen without stopping finalization/production? I think testing that, and again later with other nodes, makes sense.

I would think 2, leaving 7 online 7/9 > 2/3

Worth testing of course to be sure.

mnaamani avatar Oct 10 '22 13:10 mnaamani

After research done by @ignazio-bovo it has been determined that setting sudo key to "zero" is a bad idea, and we should revert to original plan to do a runtime upgrade to drop the sudo pallet instead.

So transition from "supervised" to "liberated" is a runtime upgrade.

mnaamani avatar Oct 10 '22 13:10 mnaamani

After research done by @ignazio-bovo it has been determined that setting sudo key to "zero" is a bad idea, and we should revert to original plan to do a runtime upgrade to drop the sudo pallet instead.

oh wow, where is conclusion noted?

Guess we need to update plan tomorrow and make some new issues for additional work.

bedeho avatar Oct 10 '22 19:10 bedeho

After research done by @ignazio-bovo it has been determined that setting sudo key to "zero" is a bad idea, and we should revert to original plan to do a runtime upgrade to drop the sudo pallet instead.

oh wow, where is conclusion noted?

https://github.com/Joystream/joystream/issues/4344#issuecomment-1274448039 cc @ignazio-bovo did you want to share any additional points?

mnaamani avatar Oct 11 '22 10:10 mnaamani