smt icon indicating copy to clipboard operation
smt copied to clipboard

[Exploration] Using Celestia's SMT for Pocket Network V1?

Open Olshansk opened this issue 1 year ago • 6 comments

@musalbas This PR isn't ready for review (yet) but I was hoping to use it as a starting point for a conversation.

After reading through https://github.com/cosmos/cosmos-sdk/issues/7100 as well as many of the downstream threads, it seems that using a variation of Libra's JMT is a good alternative to IAVL. I summarized a few of it's highlights here.

We're hoping to use a go native implementation and I very much want to avoid building out own from scratch. I came by this library and would prefer to help build on top of it together. It's easy to use, but I'm still trying to make my way through the code and understand the inner workings.

Specifically, my main questions at the moment are:

  1. The JMT paper describes a general purpose Addressable Radix Merkle Tree with a variable r, but it seems this library only support binary trees. 1.1. Is that correct? 1.2 Is there a plan to extend it?

  2. Is this library being used anywhere? 2.1. I'm primarily asking to in relation to it's performance and security "in the wild". 2.2. Would you recommend an alternative library given that it's been a couple of years?

  3. I personally found it hard to understand what "side nodes" and "path nodes" represent, so I tried to build a little visualizer. After running go test -v ./... -run TestVisualizationHelper, I created the output below, but also used a small python notebook to visualize the tree as well. 3.1 The textual output says that the side nodes of C are SIDE NODES: [ D, J, A, ZERO, ZERO, NODE, NODE ], but the tree visualization shows it at a lower height. I believe there's simply a gap in my understanding here so the main question I have is what "side nodes" are really meant to represent?

Happy to jump on a call if you think it'll be more productive than a text back & forth!

output

Leaf: F (3 side nodes, 4 path nodes)
	Largest common prefix: 2
	Common Bits : 1 0
	SIDE NODES: [ H, NODE, NODE ]
	PATH NODES: [ F, NODE, NODE, NODE ]
Leaf: G (2 side nodes, 3 path nodes)
	Largest common prefix: 1
	Common Bits : 0
	SIDE NODES: [ NODE, NODE ]
	PATH NODES: [ G, NODE, NODE ]
Leaf: J (6 side nodes, 7 path nodes)
	Largest common prefix: 5
	Common Bits : 1 1 0 1 0
	SIDE NODES: [ NODE, A, ZERO, ZERO, NODE, NODE ]
	PATH NODES: [ J, NODE, NODE, NODE, NODE, NODE, NODE ]
Leaf: A (5 side nodes, 6 path nodes)
	Largest common prefix: 4
	Common Bits : 1 1 0 1
	SIDE NODES: [ NODE, ZERO, ZERO, NODE, NODE ]
	PATH NODES: [ A, NODE, NODE, NODE, NODE, NODE ]
Leaf: B (5 side nodes, 6 path nodes)
	Largest common prefix: 4
	Common Bits : 0 1 0 0
	SIDE NODES: [ E, ZERO, I, G, NODE ]
	PATH NODES: [ B, NODE, NODE, NODE, NODE, NODE ]
Leaf: E (5 side nodes, 6 path nodes)
	Largest common prefix: 4
	Common Bits : 0 1 0 0
	SIDE NODES: [ B, ZERO, I, G, NODE ]
	PATH NODES: [ E, NODE, NODE, NODE, NODE, NODE ]
Leaf: H (3 side nodes, 4 path nodes)
	Largest common prefix: 2
	Common Bits : 1 0
	SIDE NODES: [ F, NODE, NODE ]
	PATH NODES: [ H, NODE, NODE, NODE ]
Leaf: I (3 side nodes, 4 path nodes)
	Largest common prefix: 2
	Common Bits : 0 1
	SIDE NODES: [ NODE, G, NODE ]
	PATH NODES: [ I, NODE, NODE, NODE ]
Leaf: C (7 side nodes, 8 path nodes)
	Largest common prefix: 6
	Common Bits : 1 1 0 1 0 0
	SIDE NODES: [ D, J, A, ZERO, ZERO, NODE, NODE ]
	PATH NODES: [ C, NODE, NODE, NODE, NODE, NODE, NODE, NODE ]
Leaf: D (7 side nodes, 8 path nodes)
	Largest common prefix: 6
	Common Bits : 1 1 0 1 0 0
	SIDE NODES: [ C, J, A, ZERO, ZERO, NODE, NODE ]
	PATH NODES: [ D, NODE, NODE, NODE, NODE, NODE, NODE, NODE ]

Olshansk avatar Oct 24 '22 02:10 Olshansk