Explore if we want to store inner nodes (in memory) too

[liamsi]

See discussion: https://github.com/lazyledger/nmt/pull/16#issuecomment-762362869

[liamsi]

Revisiting this issue: we are already able to store the inner nodes on ipfs/the ipld merkle dag via that visitor I've added (see for instance the PRs that Evan added to lazyledger-core: https://github.com/lazyledger/lazyledger-core/pull/178 and specifically this visitor allows us to create a batch of ipld nodes, that on calling Commit on the batch are persisted locally and can be requested.

Of course, this is not in memory but storing the nodes in memory would simply be an optimization that would allow not to recompute the inner hashes each time. This is not really urgent and could be done in combination with #15 .

[musalbas]

This is probably more important now in the context of celestia-node's storage optimizations roadmap. As we don't want to store proofs on-disk, we will want to cache them in-memory. cc @Wondertan

[Wondertan]

For some reason I assumed that is already the case. We will definitely need NMT proofs caching to be provided by nmt lib once we start moving away from IPLDv0 based proofs and caching.

[musalbas]

Right now NMT caches the leaf hashes, but not inner nodes.

[Wondertan]

Related: https://github.com/celestiaorg/rsmt2d/issues/260#issuecomment-1668164236

We would need to ensure that rsmt2d.Tree and NMT wrapper are compatible with proofs caching, as we don't interact with NMT directly.