Add BlockDetailLevel

[mangas]

Firehose block detail level introduced a source of non deterministic behavior. This PR ensures that if Call or block handlers are used, the incoming blocks are required to use Extended Block Details, this essentially means rpc poller based chains will not be allowed as they currently introduce some behaviors for which the graph-node cannot ensure deterministic subgraph data.

• Expose the existing DetailLevel information to graph-node's protobuf
• Enforce a minimum value based on the handler type
• Add env var to disable this behavior, in some changes extended block types are not yet available, so for those chains the new env var needs to be used.

Optimism at the time of writing is still a WIP. Arbitrum has a mix of block detail level and may require the env var to be used to bypass validation.

[mangas]

@sduchesneau could you please confirm if there are any additional chains that also should have the block detail validation disabled by default?

[matthewdarwin]

FWIW I could (and have) run rpc-poller on chains that support full blocks. (We have a few today we're converting to full blocks, but initially were set up with RPC poller blocks because full firehose support was not available).

Shouldn't this better be a property in the graph-node config file about what attributes the provider supports? (Like we have traces=true on and RPC node and other properties today)

Any OP-STACK or Polygon-ZK type chain does not support full blocks currently.

[mangas]

FWIW I could (and have) run rpc-poller on chains that support full blocks. (We have a few today we're converting to full blocks, but initially were set up with RPC poller blocks because full firehose support was not available).

Shouldn't this better be a property in the graph-node config file about what attributes the provider supports? (Like we have traces=true on and RPC node and other properties today)

Any OP-STACK or Polygon-ZK type chain does not support full blocks currently.

IMO This is an implementation detail and should be presented by the firehose/substreams endpoint. In terms of adding to the config, I think that approach can easily cause the same issues we're trying to prevent with this implementation which is a lot more robust.

I'll update to add the polygon-zk stuff as well

[matthewdarwin]

IMO kinda hidden environment variables that change graph-node behaviour for specific chains are hard to understand when a node operator is setting up new chains. Forcing them to think about it is better than trying to debug why things go wrong mysteriously.

Or alternatively, what is the maintenance plan to keep this ENV up-to-date as studio keeps adding new chains? Recently they are adding 4 chains per week or so.

[mangas]

This is still being discussed internally, we probably will need to sync with the chains in test and enable the full block requirement once they go live. Ideally for operators this should be mostly transparent if the sync works well, they exist in case it doesn't

[matthewdarwin]

Asside: As I brought up during recent off-site, I would ideally like to get away from forcing graph-node restart to add/remove/change a chain configuration. There are 100s of new chains launching in the next year or so.... Forcing a graph-node restart is annoying. So not adding new environment variables would be nice. (During the discussion Adam/David mentioned this would not be easy lift to accomplish).

[mangas]

Asside: As I brought up during recent off-site, I would ideally like to get away from forcing graph-node restart to add/remove/change a chain configuration. There are 100s of new chains launching in the next year or so.... Forcing a graph-node restart is annoying. So not adding new environment variables would be nice. (During the discussion Adam/David mentioned this would not be easy lift to accomplish).

I don't really see that as related to this situation, that overall seems to be like a low return investment. The extra complexity to avoid restarts is a lot of code that needs to be written for a low impact achievement (no restart) but that's just my opinion and not really related to this PR

[github-actions[bot]]

This pull request hasn't had any activity for the last 90 days. If there's no more activity over the course of the next 14 days, it will automatically be closed.