[Feature Request]: Comprehensive proof API

[damiannolan]

Overview

Use case

As an on-chain lightclient I would like to verify blob inclusion against the dataHash (data availability root) tracked in celestia-core commit headers.

For example, given a blob.Commitment in namespace NS I would like to query some RPC endpoint at height H which returns a proof veriable against the data availability root.

Problem defintion

As it exists today a blob proof queried from celestia-node's blob.GetProof api is not verifiable against the celestia data availability root signed by the validator set. The proof returned - blob.Proof is alias of []*nmt.Proof (ref).

This proof(s) is intended to be verified via an RPC call to celestia-node's Included api. Unfortunately, this does not satisfy the requirements of an on-chain lightclient, where a proof should be submitted and verified within the state machine against the data availability root.

Furthermore, namespaced merkle tree (nmt) proofs are only verifiable against their corresponding RowRoots (ref). And a collection of RowRoots for a block at a particular height N are merklized to produce the data availability root (ref). As I understand the data availability header (DAH), containing Row and Column roots of the da square is discarded by the celestia application in PrepareProposal, only storing the hash (root) in the commit header.

This leads me to believe that as an off-chain actor (e.g. ibc relayer), I would be forced to additionally query the ExtendedHeader and manually work out and reconstruct a ShareProof which is veriable against the DA root.

Or alternatively to manually work out the share indices for a particular blob and rely on the ProveShares rpc offered by celestia-core. I believe this would also require querying additional info to retrieve data such as:

• txIndex
• blobIndex
• block for txs

Some sample go code might look something like:

version := blockRes.Block.Header.Version.App
maxSquareSize := appconsts.SquareSizeUpperBound(version)
subtreeRootThreshold := appconsts.SubtreeRootThreshold(version)
blobShareRange, err := square.BlobShareRange(
	blockRes.Block.Txs.ToSliceOfBytes(),
	int(tx.Index),
	int(blob.Index),
	maxSquareSize,
	subtreeRootThreshold,
)

// get the proof of the shares containing the blob provabable to the data root
sharesProof, err := trpc.ProveShares(ctx, uint64(tx.Height), uint64(blobShareRange.Start), uint64(blobShareRange.End))
if err != nil {
	panic(err)
}

Proposal

As explained to me by @renaynay an RPC call to blob.GetProof rather returns a proof for an entire namespace. As far as I know this can be seen in the retrieve method of blob/service.go in celestia-node.

From what I understand about the celestia architecutre, a more comprehensive and cohesive proof API may look something like the following:

• GetShareProof(height, startShare, endShare uint64)
• GetBlobProof(height uint64, namespace share.Namespace, commitment blob.Commitment)
• GetNamespaceProof(height uint64, namespace share.Namespace)

As a user I would expect each of these proofs to be veriable against the data availability root. Due to my limited knowledge of the system this may not be possible but from a user's perspective I would expect this to be a desireable feature or property of the proof APIs offered by celestia.

Feel free to close this issue if it is duplicate or if you could point me another issue or resource with work or discussion about this topic I would be very interested!

[rach-id]

@damiannolan Check this PR: https://github.com/celestiaorg/celestia-node/pull/3470, I guess it solves what's needed in this issue. Your feedback is appreciated

[damiannolan]

Awesome! Thank you @rach-id, I'll check it out

[Wondertan]

@walldiss, we have most of the requested features here atm, the only thing missing to close this is making a proper blob proof, rather than a full namespace proof

[walldiss]

It does not seem like proper blob proofs would be enough. We need to make all proofs to be to dataroot. But good news are it is wip:

• https://github.com/celestiaorg/celestia-node/issues/4308

For the context we are going to have dataroot proofs in share and blob API quite soon with completion of this EPIC

[damiannolan]

That's nice to hear. As it stands you need to work out share indices as far as I understand. It would be so much simpler to just provide the blob commitment. Something like this (in rust for example):

let eds_size = header.dah.row_roots().len() as u64;
let ods_size = eds_size / 2;
let first_row_index = blob.index.unwrap() / eds_size;
let ods_index = blob.index.unwrap() - (first_row_index * ods_size);

let range_response = client
    .share_get_range(&header, ods_index, ods_index + blob.shares_len() as u64)
    .await?;

let share_proof = range_response.proof;
share_proof.verify(header.dah.hash())?;

[walldiss]

@damiannolan The issue I linked has more ambitious goal. New API will have only dataroot proofs and you no longer need would need to know about DAH or row roots or eds size. It will be more simple like this:

let datahash = header.datahash()
let height = header.height()
let blob_response = client
    .get_blob(datahash, height,blob.commitment())
    .await?;

let share_proof = range_response.proof;
share_proof.verify(datahash)?;

And similar to namespaceData or Range request/response. No DAH. New API will be available in go first and Rust implementation will follow.

[rach-id]

Just to confirm, the current proof can be used for both, share proof and commitment proof. You just need to get either the shares or the subtree roots to use them to verify the proof.

This issue aims to keep this functionality, and just add the row roots to the proof?