mainnet-fork with multiple upstream providers

[bout3fiddy]

Overview

Provide a simple overview of what you wish to see added. Please include:

• What you are trying to do

I would like to be able to do the following:

ape test --network ethereum:mainnet-fork:hardhat:geth
...

and

ape test --network ethereum:mainnet-fork:hardhat:alchemy

• Why Ape's current functionality is inadequate to address your goal This makes it easier for one to use their local node for local tests and CICD tests with alchemy

Specification

Describe the syntax and semantics of how you would like to see this feature implemented. The more detailed the better!

Remember, your feature is much more likely to be included if it does not involve any breaking changes.

ape-config.yaml would allow more flexbility in choosing upstream provider for mainnet-fork, if one could set up their configurations in the following manner:

plugins:
  - vyper
  - alchemy
  - hardhat

ethereum:
  default_network: mainnet-fork
  mainnet_fork:
    default_provider: hardhat

hardhat:
  default_upstream_provider: geth
  geth:
    fork:
      ethereum:
        mainnet
  alchemy:
    fork:
      ethereum:
        mainnet
           
geth:
  ethereum:
    mainnet:
      uri: http://localhost:9090

Dependencies

Include links to any open issues that must be resolved before this feature can be implemented.

I don't foresee any open issues blocking this, but it looks like a breaking change to me.

[fubuloubu]

This is one idea we had but in a slightly different way, basically provide fail-over functionality with how providers are chosen, so instead of one default that it tries to use or fails, you describe a preference order and it will keep trying for a connection down your preference list until it gets one (or times out)

This way, you can spend less time configuring your providers, and just describe what networks you want to connect to instead. For fork mode, upstream providers would be a subset of that list for it to cycle through.

How does this sound?

[bout3fiddy]

This is a pretty good idea. Makes CICD and local development with a hosted node seamless. Can you point me to the modules that need to be changed to implement this feature?

[fubuloubu]

This is a pretty good idea. Makes CICD and local development with a hosted node seamless. Can you point me to the modules that need to be changed to implement this feature?

It would be more on the config classes for each ecosystem/network, and some more extensive work to NetworkManager and likely ProviderContextManager as well (since that actually implements the switching logic)

[bout3fiddy]

Hmm so I imagine the config would look something like this:

plugins:
  - vyper
  - alchemy
  - hardhat

ethereum:
  default_network: mainnet-fork
  mainnet_fork:
    default_provider: hardhat

hardhat:
  port: auto
  fork:
    ethereum:
      mainnet:
        upstream_provider: geth
        upstream_provider: alchemy

geth:
  ethereum:
    mainnet:
      uri: http://localhost:9090

Ape first tries geth. if connection borks, then ape tries alchemy.

[fubuloubu]

More accurate:

hardhat:
  fork:
    ethereum:
      mainnet:
        upstream_provider: 
        - geth
        - alchemy