Expose `graphman` commands through `graph-node`'s admin GraphQL interface

[tilacog]

We could expose all graphman's commands as GraphQL mutations over the indexer management API endpoint (port 18000, by default).

In this design, graph-node would act as a service, exposing all necessary routes and returned types for other tools to interface with.

This concept is similar to docker, which is implemented as a daemon.

[chriswessels]

This is how the indexer components work. indexer-agent exposes an indexer management GraphQL API, and indexer-cli presents a CLI interface over this strongly typed and inspectable API.

[lutter]

The main reason why graphman doesn't work that way is authentication: with the CLI tool, authn is punted to the config file where you proved that you can get to the DB passwords. With an API, we'd need to make sure that that API is not accessible to unauthorized parties, especially since graphman misuse can be catastrophic for an indexer (e.g., delete all subgraphs)

BTW. docker is famous for being very hard to secure properly; security is a big part of the reason why Red Hat switched to podman instead of the upstream docker daemon

[chriswessels]

In both instances, there's a client and a server. In both instances, the connection between them must be secured.

Today the client is graphman and the server is PostgreSQL. With this proposal, the server becomes a graph-node instance (which in turn writes to Postgres). IMO this is much easier to secure than access to the database. This is also more reliable in the event that the connection between the client and the server is spotty.

Moving to an API approach also makes automating the tasks graphman does much easier. Instead of spawning an executable (graphman) that has access to the database and its password, you could interact with an API that provides a constrained set of management tasks and an independent auth mechanism that doesn't risk leaking the DB pass.

[github-actions[bot]]

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.

[YassinEldeeb]

RFC: Integration of Graphman Commands into GraphQL Server for Graph-node

Summary

This RFC proposes the integration of core functions from the Graphman CLI into a new GraphQL server for Graph-node that runs on port 8050 spawned as a child process of graph-node's main process

The core functions, which have been successfully extracted and reused across the CLI can be reused within the GraphQL server.

Integrating the stats subcommands as a starting point will be useful:

• account-like: Toggle whether a table is account-like
• analyze: Perform a SOL ANALYZE in a Entity table
• set-target: Print this message or the help of the given subcommand (s)
• show: Show statistics for the tables of a deployment
• target: Set the statistics targets for the statistics collector

These functions are essential for the operation of the Graphman CLI and they provide valuable capabilities for the GraphQL server.

Motivation

The graph-node project offers several functionalities via the CLI tool graphman. However, for broader accessibility and ease of use, it's necessary to expose these capabilities through a GraphQL server. By doing so, users can interact with deployments, manage subgraphs, and handle block and cache data more seamlessly, without directly dealing with the CLI, thereby taking full advantage of the features and benefits offered by GraphQL.

Detailed Design

The detailed design will follow a four-step process:

1. Identifying shared logic

The shared logic between the CLI and GraphQL server will be identified and will be the focus of the integration. This includes understanding how these functions interact with other parts of the application.

2. Function extraction

Once the shared logic has been identified, the next step is to extract these functions from the CLI into a format that can be implemented into the GraphQL server.

3. Function implementation

After extraction, the next step is to implement these functions into the GraphQL server. This will involve understanding the GraphQL server's structure and designing a way to integrate these functions smoothly.

4. Testing and deployment

After implementation, rigorous testing will be carried out to ensure that the integration of the core functions is successful and has not introduced any new bugs.

Documentation

We will need to update the documentation to emphasize everywhere where Graphman is introduced or used that there's also an API to use these core functions within the GraphQL server. Examples of usage and troubleshooting tips will be included.

Drawbacks

There might be some challenges during the integration process, such as compatibility issues between the CLI and GraphQL server. However these can be identified and resolved as we progress.

Alternatives

An alternative would be to recreate these functions from scratch within the GraphQL server. However, this could be more time-consuming and could introduce unnecessary redundancy.

Unresolved Questions

• Are there any other functions in the Graphman CLI that would also be beneficial if integrated into the GraphQL server after the stats subcommands that are the most important at the moment?
• How are we planning to handle long-running commands within the GraphQL context? Job queue?

Conclusion

This proposal to integrate the core functions of Graphman from the CLI into the GraphQL server of Graph-node will extend the capabilities of the server and make it more versatile and powerful. This will enhance the user experience and provide more functionality to the developers.

This is a start of negotiating and having a conversation on next steps. This pull request has a proof of concept with a working graphql API that can be tested, and will be built on top of it. https://github.com/graphprotocol/graph-node/pull/4598

[YassinEldeeb]

@lutter @fordN would love to discuss here

[azf20]

hey @YassinEldeeb - this seems reasonable and could unlock new interfaces and quality of life improvements for indexers. Will this first effort lay some groundwork for simpler & swifter extension to the remaining graphman commands?