graph-node icon indicating copy to clipboard operation
graph-node copied to clipboard

Support parameterized Substreams modules from the subgraph.yaml manifest

Open abourget opened this issue 1 year ago • 0 comments

Description

There is already support for applying params in https://github.com/graphprotocol/graph-node/pull/4759/files but I'm not sure it's exposed to the end users.

I see no reference to it in https://thegraph.com/docs/en/cookbook/substreams-powered-subgraphs/#defining-a-substreams-package and I've been told that it was used internally, but not exposed externally by @mangas

With all the efforts to simplify using Substreams, it would be an incredible addition that we can use some well-known (we're calling them Foundational) modules can be reused, and provide a semantic for querying the chain's data, using indexes to skip lots of blocks, etc..

dataSources:
  - kind: substreams
    name: Transaction
    network: mainnet
    source:
      package:
        moduleName: map_filter_transactions
        file: ethereum-explorer-v0.1.2.spkg
        params:
          map_super_bob: "this is a nice params string"
          "ethcommon:map_filter_logs": "this is a great params string
# the previous version is how params are laid out in `substreams.yaml`, a little more terse, but some might dislike the quotes around imported modules, so
# alternatively:
        params:
          - module: map_super_bob
            value: "this is a nice params string"
          - module: ethcommon:map_filter_logs
            value: "(contract:0x23123123123)"

Validation that would be required, to ensure consistency:

  • the module must exist in the corresponding file's .spkg
  • that module must have, as its first input, a sf.substreams.v1.Param type

only then, do we apply the params to the module.

/cc @saihaj we'd need support in graph-cli first, so it doesn't get published if some of that validation fails.. but perhaps we don't tweak the spkg upon publishing (keeping the same IPFS reference for all), and only applying the params down into graph-node. That's probably cleaner.

abourget avatar May 29 '24 20:05 abourget