feat: Support multi-validator architecture

[jsimnz]

Resolves #2473

This is an initial draft PR to get feedback. Still, some rough edges that will be worked on, but this PR is already late enough relative to when the issue/bounty was issued, and I know there's interest from others, so hopefully this moves the ball forward.

This is a fairly significant (breaking) change to the chain service, configuration, and folder structure of the chain home directory.

You can see the comment thread on #2473 for some of the ideas / changes that are implemented.

TLDR

• Support multiple validators on the configuration
• Deprecate old config fields that are superceded by the validator specific config.
• Uses the same Go command runner to start multiple validators.
• Several changes to the chain home directory.
  • Each validator has its own home-folder which contains the respective config/data folders
  • Shared keyring folder used by all validators
  • Single genesis.json that is symlinked into each validator config folder
  • The "root" validator (ie the first validator in the config array) is symlinked into the root config folder (see diagram below).

The various symlinks are used to keep a nice developer/user experience. For example, symlink the config files from the main validator into the root config folder means we can still use the same CLI operations marsd start which will start a single validator, or ignite chain serve which will start all the validators.

[jsimnz]

@ilgooz any thoughts on this? I know its a decently requested feature, would love to get some feedback on this so we can move the ball forward to eventually getting it merged :)

[aljo242]

@jsimnz we are prioritizing efforts to integrate this work! Would you like to resolve all of the conflicts? Otherwise, we can do this.

[jsimnz]

@jsimnz we are prioritizing efforts to integrate this work! Would you like to resolve all of the conflicts? Otherwise, we can do this.

Happy to resolve them. Was just twiddling my thumbs hoping to get some eyes on this. Glad there's some movement 👍

[jsimnz]

@aljo242 Would you prefer I use the new config version system, as there is some duplicate work in the original version of this PR, compared to the new config version (since it wasn't implemented when I first started on this feature).

I assume I should use the new config types (since they are merged), just wanted to confirm.

[aljo242]

@aljo242 Would you prefer I use the new config version system, as there is some duplicate work in the original version of this PR, compared to the new config version (since it wasn't implemented when I first started on this feature).

I assume I should use the new config types (since they are merged), just wanted to confirm.

Yes, the new config system is what we should be using!

[aljo242]

@jsimnz any update?

[jsimnz]

Sorry, got caught up with regular work tasks. Happy to bring this up to parity with development. Unfortuently Ive originally duplicated all the config changes, so just need to rip that part out so I can use the new system. Will take a short bit of time, will keep things updated.

[aljo242]

Sorry, got caught up with regular work tasks. Happy to bring this up to parity with development. Unfortuently Ive originally duplicated all the config changes, so just need to rip that part out so I can use the new system. Will take a short bit of time, will keep things updated.

Sounds great! Let us know if we can help in any way.

[fadeev]

@aljo242 we might want to reopen this one as well.

[aljo242]

@jsimnz this PR was automatically closed when the develop branch was deleted. Could you re-open this and target the main branch?

Please let us know if we can help with merging.

[jsimnz]

@aljo242 thanks for the ping, can happily re-open. Made some great progress rebasing to develop