Represent IPFS Cluster in WebUI/Desktop

[olizilla]

ETA April 2020: Let's also consider the "pinning buddy system" thoughts originally in https://github.com/ipfs/ipfs-gui/issues/36 as one way to introduce Cluster concepts.

@hsanjuan made the good point that the IPFS Web UI could usefully show info about IPFS Cluster nodes if the api is available.

As we discussed about having a Cluster tab in the IPFS web UI, here are some pointers and thoughts.

• Every cluster peer exposes their own REST API endpoint (at tcp/9094 by default)
• GET /id shows Cluster peer information
• GET /peers gives an array of [/id] for every cluster peer
• API docs: https://cluster.ipfs.io/developer/api/
• Someone wrote a cluster api client already (not sure if it's complete): https://github.com/te0d/js-ipfs-cluster-api
• Regarding response formats:
  • /id /peers return https://godoc.org/github.com/ipfs/ipfs-cluster/api#IDSerial
  • /pins returns [] of https://godoc.org/github.com/ipfs/ipfs-cluster/api#PinInfoSerial
  • /allocations returns [] of https://godoc.org/github.com/ipfs/ipfs-cluster/api#PinSerial

Things we need

• [ ] Support CORS Headers in Cluster for browsers to be happy showing stuff from here.
• [ ] See my list of peers with their names and IDs (or a fancy diagram).
• [ ] Display the pin statuses (/pins), filter, sort them etc. CIDs would have links to the IPLD explorer.

...and more input!

• What extra info does cluster expose that we'd benefit from visualising?
• What blockers are there to re-using the new Web UI with cluster today?

[lidel]

Here's an idea: how about making "Cluster UI" a proof-of-concept of the "WebUI Add-on Apps" Juan mentioned in recent conversations?

[olizilla]

One couple of milestones for this could be

• Web UI can show cluster info
• Cluster gets its own web app that is focused on the needs of a sys-admin managing a professional set up

The current Web UI would remain focused on on-boarding new users to IPFS; the cluster UI could become the sys-admins dashboard for monitoring network conditions, managing access control, and visualising the balance of data across the nodes.

[olizilla]

Also of note, cluster could be a good place to experiment with a streaming / websocket / graphql based api so we can build fancy live updating UIs without polling a restful api.

[hsanjuan]

Hello, I will add CORS support for the next release.

The cluster page should show:

• From /id
  • Current peer (name and peer ID)
• From /allocations
  • List of pins and names and replication factors

The two above are local-peer commands. Non local peer commands extract information from other peers and may take longer to return if one of them is hanging:

• From /peers: information about other peers in the cluster
• From /pins: pin status list (same items as /allocations), with information about the item being pinned or pinning or error or remote in each peer. This may be an expensive call on very large pinsets.

Both /pins and /allocations support requesting single-item information by attaching /<cid> to the path.

I'm not sure if this should all be in a single tab, or in several. Probably it makes sense to be able to see the list of allocations and then if the user clicks on them show all the information for that CID.

[hsanjuan]

Also of note, cluster could be a good place to experiment with a streaming / websocket / graphql based api so we can build fancy live updating UIs without polling a restful api.

For a while I wanted to have a streaming event API but that's going to need some time. It would be faster to provide a websocket API though, which just behaves more or less the same but taking request on a websocket connection. I don't know if this has any advantages though.

[hsanjuan]

Hey all, Cluster 0.7.0 added [CORS] headers support to the API, so it should be possible to put the web ui on top and have it work nicely.

[vasa-develop]

Hey! I was working on something similar. Here is a POC of what I am building.

[jessicaschilling]

Linking here to (closed) issue https://github.com/ipfs/ipfs-gui/issues/36 for thoughts on incorporating collaborative cluster nodes in our GUIs. Representing it as a "pinning buddy system" may be a good way to introduce the concept to those who aren't familiar with Cluster.

[jessicaschilling]

Note recent https://github.com/ipfs-shipyard/ipfs-desktop/issues/1648 for request for this functionality. Revisiting after pinning service integration work is complete might be a good path forward.

[jessicaschilling]

Cross-referencing https://github.com/ipfs/roadmap/issues/85.