Usability/DevUX Meta Issue

[liamsi]

This issue collects smaller quirks with the current API/docs/CLI in one place. I will list them here and break them out into their own issues where it makes sense:

• [ ] the comment here is wrong: https://github.com/celestiaorg/celestia-node/blob/f2b664f3abe5e76ed51e19305aa3489980a93692/blob/service.go#L113 (it only returns one blob; GetAll is what returns all blobs)
• [ ] GetAll is confusing and edge-cases are under-specified: https://github.com/celestiaorg/celestia-node/issues/3292#issuecomment-2059313529
• [ ] some examples in API docs are not representative; e.g. blob.Get returns a different commitment than requested (#3317 )
• [ ] CLI: when looking at the for usage returned by --help encoding related guidance is often missing, eg. celestia blob get [height] [namespace] [commitment] [flags] should state which format the inputs are expected in (hex? base64?). The CLI help should be self-contained; currently one needs to e.g. look here to know how to use this
• [ ] CLI: namespaces are submitted as hex but returned base64 encoded (which is very confusing); example:

celestia blob submit 0x42690c204d39600fddd3 0x676d                                                                                                                  
{
  "result": {
    "height": 1671475,
    "commitments": [
      "0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY="
    ]
  }
}
⋊> ~/G/celestia-node on main ⨯ celestia blob get-all 1671475  0x42690c204d39600fddd3                                                                                                               12:06:50
{
  "result": [
    {
      "namespace": "AAAAAAAAAAAAAAAAAAAAAAAAAEJpDCBNOWAP3dM=",
      "data": "0x676d",
      "share_version": 0,
      "commitment": "0MFhYKQUi2BU+U1jxPzG7QY2BVV1lb3kiU+zAK7nUiY=",
      "index": 531
    }
  ]
}

• [ ] CLI/API: the node daemon gets started with only one account and one can only submit PFBs from that single account; this does not reflect how people interact with chains using different wallets for different purposes (applies to e.g. celestia blob submit and a bunch of state related commands / rpcs) https://github.com/celestiaorg/celestia-node/issues/3295
• [ ] #3345
• [ ] consumers of the API do not have sufficient control for retries on blob submission
• [ ] also related to the above: https://github.com/celestiaorg/celestia-node/issues/3164
• [ ] #3361
• [ ] API: Clarify State.SubmitPayForBlob vs Blob.Submit: both methods are too similar and it's unclear when to use which (related: https://github.com/celestiaorg/celestia-node/pull/3349#issuecomment-2109995753)

[Bidon15]

Should be populated in node somehow

Blobstream APIs in node

https://github.com/celestiaorg/celestia-core/issues/1306

Many Execution Environments to 1 DA Node

https://github.com/celestiaorg/celestia-app/issues/3259 https://github.com/celestiaorg/celestia-node/issues/3295 https://github.com/celestiaorg/celestia-node/discussions/3022

Separation of endpoints for DA Node (RPC/GRPC)

https://github.com/celestiaorg/celestia-node/issues/2931

JWT Secret Rotation

https://github.com/celestiaorg/celestia-node/issues/3293

[tuxcanfly]

I would like to add a couple of improvement ideas for reliable blob submission which is a blocker for rollup liveness:

• [ ] "rpc error: code = Unknown desc = timed out waiting for tx to be included in a block" error is returned frequently possibly due to 1) mempool congestion 2) big blobs 3) too small timeout_broadcast_tx_commit
• [ ] ": tx already in mempool" always happens immediately after the above error and requires waiting out until the tx drops from the mempool
• [ ] ": incorrect account sequence" occasionally happens in place of the above error but also requires waiting out until the tx drops from the mempool
• [ ] rpc error: code = Unknown desc = error on broadcastTxCommit: tx size is too big: 1962442, max: 1962441 happens when submitting big blobs due to padding shares even though the supposed limit is 1974272 so this breaks the contract with node.
• [ ] rpc error: code = Unknown desc = error on broadcastTxCommit: tx too large occasionally big blobs return this in place of the above. I see that there's also Tx too large. Max size is %d, but got %d in the code.

Since there is no unique error code for these errors, we're currently grepping the string for these error strings which is not ideal.

It would be good to have a standardised error list and unique error codes and if possible, a error severity, and hint to allow clients to retry gracefully.

[MSevey]

Related: https://github.com/rollkit/go-da/issues/65

[vgonkivs]

https://github.com/celestiaorg/celestia-node/pull/3537

[vgonkivs]

also related to the above: https://github.com/celestiaorg/celestia-node/issues/3164

After the fix on the app the issue "invalid account sequence" gone.

[vgonkivs]

the comment here is wrong

https://github.com/celestiaorg/celestia-node/pull/3530

GetAll is confusing and edge-cases are under-specified

https://github.com/celestiaorg/celestia-node/pull/3223

some examples in API docs are not representative; e.g. blob.Get returns a different commitment than requested

https://github.com/celestiaorg/celestia-node/pull/3479

CLI: when looking at the for usage returned by --help encoding related guidance is often missing, eg. celestia blob get [height] [namespace] [commitment] [flags] should state which format the inputs are expected in CLI: namespaces are submitted as hex but returned base64 encoded (which is very confusing); example:

https://github.com/celestiaorg/celestia-node/pull/3553 https://github.com/celestiaorg/celestia-node/pull/3537

CLI/API: the node daemon gets started with only one account and one can only submit PFBs from that single account; this does not reflect how people interact with chains using different wallets for different purposes (applies to e.g. celestia blob submit and a bunch of state related commands / rpcs

https://github.com/celestiaorg/celestia-node/pull/3349

https://github.com/celestiaorg/celestia-node/issues/3345

https://github.com/celestiaorg/celestia-node/pull/3391

API: Clarify State.SubmitPayForBlob vs Blob.Submit: both methods are too similar and it's unclear when to use which

https://github.com/celestiaorg/celestia-node/pull/3349#issuecomment-2109995753