Flow API Routing Calls to DPS or BDS

[franklywatson]

@AliSMAmin commented on Wed May 18 2022

Context

Related to issue #942. This epic pertains to the Flow API Service, a part of the wider Flow Access Node Refactoring.

Proposed Solution

The Flow API Service hosts the present-day Access API as a standalone process, decoupling it from the current Access Node. The Flow API Service has the primary responsibility of handling and routing requests to the upstream components, either Data Provisioning Service (DPS) or Blockchain Data Service (BDS).

In the longer term this service will tackle traffic management and optimization concerns such as rate-limiting, caching, spam prevention, QoS, and possibly more. However, these extended capabilities remain outside the scope of this task.

Step Definitions

• The API Service implements the Flow API (aka Access API), using an upstream DPS’s API as well as BDS’s Protocol and Execution State APIs
• Since the DPS API is not exposed as part of the Flow API, the Flow API Service observes the routing logic defined in [x document provided] to forward from Flow API endpoints to DPS endpoints
• Calls made to the existing methods found on the Access API are routed through to correspondingly named methods on either the Protocol or Execution state APIs provided by BDS.
• Existing Access API tests are migrated and pass on the Flow API
• The configuration that is used to indicate the upstream DPS or BDS should support the notion of
• If data is unavailable, return error

Definition of Done

• Flow API
  • As Access Node, SendTransaction is handled via SendTransaction on the Protocol API of the upstream BDS
  • As Observer service, it successfully forwards gRPC and/or HTTP SendTransaction requests to the configured upstream Access Node or Observer Service
  • It successfully handles upstream data retrieval for Protocol state
    • The following Flow API methods are routed to the BDS
      • [Provide a definitive, confirmed list of what these method mappings are]
    • The following Flow API methods do not route to the BDS and are handled by the DPS
      • [Provide a definitive, confirmed list of what these method mappings are]
  • It successfully handles upstream data retrieval for Execution state
    • The following Flow API methods are routed to the BDS
      • [Provide a definitive, confirmed list of what these method mappings are]
    • The following Flow API methods do not route to the BDS and are handled by the DPS
      • [Provide a definitive, confirmed list of what these method mappings are]
• If BDS isn't configured on bootup, return an error and exit. If DPS is not configured, only return an error for data that is unavailable (outside of current spork) when called, otherwise function with BDS. [Might there be a discovery mechanism, or it always has to be configured?]
• If BDS and/or DPS aren't connected or responding, return an error when the Flow API being called times out

---

References

DPS APIs are: https://github.com/optakt/flow-dps/blob/master/docs/dps-api.md

Protocol APIs are: https://github.com/onflow/flow-go/issues/2420

Execution Data APIs are: https://github.com/onflow/flow-go/issues/2421

[AliSMAmin]

All relevant method mappings are delineated in: https://github.com/optakt/flow-dps-access

[AliSMAmin]

We decided to stand-up two Observers in order to start this proxying work instead of bootstrapping a DPS because of internal DPS problems.

[szegedim]

Here is the passing test for this change on the Access APi: https://github.com/GetElastech/api-service/actions/runs/2776003280 @A-town21 is adding the protocol changes. We might have some design suggestions soon.

[peterargue]

we've opted to go in a different direction and instead integrate register indexing into ANs.