Indexer on Betanet automation

[chefsale]

Story

To enable fully-featured Explorer on betanet, we need to deploy Indexer for Explorer for it, but the challenge is that betanet is based on nightly builds of nearcore, and automatically wipes the data on protocol and database changes.

Implementation Notes

We have automation in place for testnet and mainnet. Most of these can be reused, the only things questionable are:

• difference in protocol version changes of nightly in betanet
• automated hardforks and wiping the database of indexer and restarts.

cc: @bowenwang1996 @mhalambek @frol @khorolets

[frol]

difference in protocol version changes of nightly in betanet

It seems that we need to automatically build indexer-for-explorer based on nearcore commit we use for betanet (patch Cargo.toml on the fly)

[chefsale]

We daily deploy master on betanet, so this would be just the tip of the master and patch that into the Cargo.toml I assume these two lines: (https://github.com/near/near-indexer-for-explorer/blob/master/Cargo.toml#L34-L35). We in that case would need to extend the betanet deployment script to:

1. Patch the lines in Cargo.toml
2. Build the release
3. Wipe the DB for betanet
4. Deploy the new binary from the patched build

Also this would all run after the betanet deployment has already happened I believe (or before)?

[khorolets]

@chefsale

automated hardforks and wiping the database of indexer and restarts.

this is a matter of a few scripts including restart of indexer

@frol as for

It seems that we need to automatically build indexer-for-explorer based on nearcore commit we use for betanet (patch Cargo.toml on the fly)

I disagree, quite often it includes some changes in the dependencies which might not be fine with diesel :(

[stefanopepe]

As a heads-up, we are reducing the footprint of the Indexer in the wallet, enabling transactions - thus the near login functionality - also on betanet. This is the wip issue: https://github.com/near/near-wallet/issues/1747

Overall, this means that even if the indexer is temporarily offline or not properly working, the web wallet will be able to provide basic functionalities for betanet-based apps and tests

[frol]

I disagree, quite often it includes some changes in the dependencies which might not be fine with diesel :(

If that breaks, it is better to be aware of it sooner rather than later. We don't strike for 100% uptime on betanet and I believe we will be good at least 80% of the time.