Allow subgraphs to be coiped/shared between indexers

[tilacog]

At first glance, this feature would operate similarly to graphman copy but subgraphs could be sent to a foreign cluster.

The motivation behind this encompasses both:

• mutual aid while indexing a given subgraph, and
• marketplace dynamics with indexed data

Alternatively, an import/export pattern similar to pg_dump would also fit those needs.

[chriswessels]

Worth noting that there are risks here. If the default pattern becomes for indexers to download existing state, then we have far less actors actively validating that state's integrity and provenance. This would also impact POI consensus assumptions.

[azf20]

I think @lutter has looked at the mechanics of copying a given subgraph previously, and if I recall correctly there were some potential challenges.

@chriswessels there are certainly those considerations, but I think the scope of this issue should focus on how such a transfer might be implemented in Graph Node, to transfer to a foreign db - those other things warrant discussion in a GIP / in the forum, and have cascading impact on other components (e.g. indexer components)

[lutter]

The main headache with import/export is defining a good format for transferring the data; it would be desirable to make that format usable for both dump/restore and for substreams. That format should be db agnostic, and live at the level of the subgraph schema, not the current db schema.

In my mind, the first order of business for this though is to address the network concerns around import/export; that might be as simple as "we don't care from the network side, we'll just treat this the same as people sharing SQL dumps" or more involved.

[github-actions[bot]]

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.

[air3ijai]

Is there a way to copy Subgraph from one indexer to another one, to bootstrap the indexing process? Can we do it somehow directly via DB or using a graphman?

Otherwise it is about a long time of indexing and load on RPC endpoint.