graph-node
graph-node copied to clipboard
Expose `graphman` commands through `graph-node`'s admin GraphQL interface
We could expose all graphman
's commands as GraphQL mutations over the indexer management API endpoint (port 18000
, by default).
In this design, graph-node
would act as a service, exposing all necessary routes and returned types for other tools to interface with.
This concept is similar to docker
, which is implemented as a daemon.
This is how the indexer
components work. indexer-agent
exposes an indexer management GraphQL API, and indexer-cli
presents a CLI interface over this strongly typed and inspectable API.
The main reason why graphman
doesn't work that way is authentication: with the CLI tool, authn is punted to the config file where you proved that you can get to the DB passwords. With an API, we'd need to make sure that that API is not accessible to unauthorized parties, especially since graphman
misuse can be catastrophic for an indexer (e.g., delete all subgraphs)
BTW. docker is famous for being very hard to secure properly; security is a big part of the reason why Red Hat switched to podman
instead of the upstream docker daemon
In both instances, there's a client and a server. In both instances, the connection between them must be secured.
Today the client is graphman
and the server is PostgreSQL. With this proposal, the server becomes a graph-node
instance (which in turn writes to Postgres). IMO this is much easier to secure than access to the database. This is also more reliable in the event that the connection between the client and the server is spotty.
Moving to an API approach also makes automating the tasks graphman
does much easier. Instead of spawning an executable (graphman
) that has access to the database and its password, you could interact with an API that provides a constrained set of management tasks and an independent auth mechanism that doesn't risk leaking the DB pass.
Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.
RFC: Integration of Graphman Commands into GraphQL Server for Graph-node
Summary
This RFC proposes the integration of core functions from the Graphman CLI into a new GraphQL server for Graph-node that runs on port 8050
spawned as a child process of graph-node's main process
The core functions, which have been successfully extracted and reused across the CLI can be reused within the GraphQL server.
Integrating the stats
subcommands as a starting point will be useful:
-
account-like
: Toggle whether a table is account-like -
analyze
: Perform a SOL ANALYZE in a Entity table -
set-target
: Print this message or the help of the given subcommand (s) -
show
: Show statistics for the tables of a deployment -
target
: Set the statistics targets for the statistics collector
These functions are essential for the operation of the Graphman CLI and they provide valuable capabilities for the GraphQL server.
Motivation
The graph-node project offers several functionalities via the CLI tool graphman. However, for broader accessibility and ease of use, it's necessary to expose these capabilities through a GraphQL server. By doing so, users can interact with deployments, manage subgraphs, and handle block and cache data more seamlessly, without directly dealing with the CLI, thereby taking full advantage of the features and benefits offered by GraphQL.
Detailed Design
The detailed design will follow a four-step process:
1. Identifying shared logic
The shared logic between the CLI and GraphQL server will be identified and will be the focus of the integration. This includes understanding how these functions interact with other parts of the application.
2. Function extraction
Once the shared logic has been identified, the next step is to extract these functions from the CLI into a format that can be implemented into the GraphQL server.
3. Function implementation
After extraction, the next step is to implement these functions into the GraphQL server. This will involve understanding the GraphQL server's structure and designing a way to integrate these functions smoothly.
4. Testing and deployment
After implementation, rigorous testing will be carried out to ensure that the integration of the core functions is successful and has not introduced any new bugs.
Documentation
We will need to update the documentation to emphasize everywhere where Graphman is introduced or used that there's also an API to use these core functions within the GraphQL server. Examples of usage and troubleshooting tips will be included.
Drawbacks
There might be some challenges during the integration process, such as compatibility issues between the CLI and GraphQL server. However these can be identified and resolved as we progress.
Alternatives
An alternative would be to recreate these functions from scratch within the GraphQL server. However, this could be more time-consuming and could introduce unnecessary redundancy.
Unresolved Questions
- Are there any other functions in the Graphman CLI that would also be beneficial if integrated into the GraphQL server after the
stats
subcommands that are the most important at the moment? - How are we planning to handle long-running commands within the GraphQL context? Job queue?
Conclusion
This proposal to integrate the core functions of Graphman from the CLI into the GraphQL server of Graph-node will extend the capabilities of the server and make it more versatile and powerful. This will enhance the user experience and provide more functionality to the developers.
This is a start of negotiating and having a conversation on next steps. This pull request has a proof of concept with a working graphql API that can be tested, and will be built on top of it. https://github.com/graphprotocol/graph-node/pull/4598
@lutter @fordN would love to discuss here
hey @YassinEldeeb - this seems reasonable and could unlock new interfaces and quality of life improvements for indexers. Will this first effort lay some groundwork for simpler & swifter extension to the remaining graphman
commands?