substrate icon indicating copy to clipboard operation
substrate copied to clipboard

Move beefy-merkle-tree to utils/binary-merkle-tree and make it generic

Open DaviRain-Su opened this issue 2 years ago • 8 comments

close: #13075

We made our own beefy light client (https://github.com/octopus-network/beefy-light-client) for the library that we also want to use in the near contract, so we want to keep beefy-merkle-tree from introducing any substrate crate here. near contract has not been compiled.

polkadot companion: https://github.com/paritytech/polkadot/pull/6528

DaviRain-Su avatar Jan 05 '23 11:01 DaviRain-Su

I don't understand the purpose of this PR. Can you please add more details to it and/or it's associated issue? Please describe the problem and the proposed solution.

Added some background description information

DaviRain-Su avatar Jan 05 '23 13:01 DaviRain-Su

1. Moving just the algorithmic part in a different crate that wouldn't have many dependencies. But seems a bit too much.

2. Conditional compilation. Defining a target that would compile only the algorithmic logic and would bring with it only the required dependencies.

Both options 1 & 2 look good to me, somewhat leaning toward option 1. It seems that the only "non-algorithmic logic" is the decl_runtime_apis! which I added here at some point, but in retrospect it's not really related with the rest of the crate. So decl_runtime_apis can move to the pallet (like this PR does) or if that doesn't work, to its own separate primitives/beefy-mmr crate.

"The algorithmic logic" is used by the beefy-mmr pallet for:

  1. building the MMR leaves that are later used by BEEFY in consensus (all BEEFY basically does is run consensus on the MMR root - thus the leaves' contents),
  2. building a BEEFY ValidatorSet as a very compact BeefyAuthoritySet: it contains just the merkle root of the tree built from validator keys as leaves.

So it is related to both BEEFY and MMR. But that's just how we use it. It can be used in many ways, so I agree it could go in its own crate independent of pallet-beefy, pallet-mmr or pallet-beefy-mmr (it will just be used by one/more of them).

On that note, I would also change its name to something without beefy, and change its location to utils/ or primitives/.

acatangiu avatar Jan 06 '23 16:01 acatangiu

On that note, I would also change its name to something without beefy, and change its location to utils/ or primitives/.

@paritytech/sdk-node wdyt? any other suggestion?

acatangiu avatar Jan 10 '23 12:01 acatangiu

On that note, I would also change its name to something without beefy, and change its location to utils/ or primitives/.

@paritytech/sdk-node wdyt? any other suggestion?

Can't speak for the whole sdk-node, but if it's not inherently beefy-specific (as in - it's a data structure that can be useful in general) nor substrate-specific then utils sounds good to me. (We already have e.g. the fork-tree in utils as a precedent.)

Just please note that there already exists a crate named merkle_tree on crates.io, which, AFAIR, will prevent a crate with the name of merkle-tree from being published. So considering that we want to publish our crates on crates.io you'll have to change the name to something else.

koute avatar Jan 10 '23 12:01 koute

On that note, I would also change its name to something without beefy, and change its location to utils/ or primitives/.

@paritytech/sdk-node wdyt? any other suggestion?

Sounds okay to me.

bkchr avatar Jan 10 '23 12:01 bkchr

@DaviRain-Su please pull master and rebase this PR

acatangiu avatar Jan 12 '23 13:01 acatangiu

@bkchr please take another look

acatangiu avatar Jan 18 '23 16:01 acatangiu

bot merge

bkchr avatar Jan 28 '23 21:01 bkchr

Waiting for commit status.

bot rebase

bkchr avatar Jan 28 '23 21:01 bkchr

Error: Command 'Command { std: "git" "push" "--porcelain" "octopus-network" "move-beefy-mmr-primitives-runtime-api", kill_on_drop: false }' failed with status Some(128); output: remote: Permission to octopus-network/substrate.git denied to paritytech-processbot[bot]. fatal: unable to access 'https://x-access-token:${SECRET}@github.com/octopus-network/substrate.git/': The requested URL returned error: 403

Merge cancelled due to error. Error: Statuses failed for 7af6cdf51b4471aa770a33841fda1bcd7aa83b92

@DaviRain-Su please merge master.

bkchr avatar Jan 28 '23 22:01 bkchr

bot merge

bkchr avatar Jan 28 '23 23:01 bkchr

Waiting for commit status.

Merge cancelled due to error. Error: Statuses failed for 9be08e305de70b87f3bf171f22091864168c4050

I don't know how to fix.

DaviRain-Su avatar Feb 01 '23 14:02 DaviRain-Su

Doesn't seem to be a problem with the PR. It's probably the CI. Not sure how to fix it myself, but I expect it to be fixed soon.

serban300 avatar Feb 01 '23 15:02 serban300

bot merge

acatangiu avatar Feb 02 '23 10:02 acatangiu