nearcore icon indicating copy to clipboard operation
nearcore copied to clipboard

NEP-364

Open blasrodri opened this issue 2 years ago โ€ข 7 comments

Implementing the following host functions according to the design proposed in NEP-364

    pub fn ed25519_verify(
        &mut self,
        sig_len: u64,
        sig_ptr: u64,
        msg_len: u64,
        msg_ptr: u64,
        pub_key_len: u64,
        pub_key_ptr: u64,
        register_id: u64,
    ) -> Result<()>;
  • [x] added unit tests
  • [x] added costs (this needs to be reviewed, since I'm not sure how to infer the right numbers)

blasrodri avatar Jul 05 '22 07:07 blasrodri

Absolutely epic work ๐Ÿ™Œ๐Ÿฟ

For tracking we're missing:

  • [ ] in-memory trie construction: https://github.com/paritytech/substrate/blob/6a946fc36d68b89599d7ca1ab03803d10c78468c/primitives/io/src/lib.rs#L515-L520

  • [ ] membership and non-membership trie verification: https://github.com/ComposableFi/composable/blob/be2aebf23aa38888a7ceba1c4a2b698500497e83/frame/ibc/src/host_functions.rs#L36-L61

seunlanlege avatar Jul 05 '22 15:07 seunlanlege

As someone with the most context here, assigning self as reviewer here. Once the PR is ready for proper review, will do a detailed review and / or delegate as needed.

akhi3030 avatar Jul 11 '22 14:07 akhi3030

@blasrodri: PTAL: https://near.zulipchat.com/#narrow/stream/295306-pagoda.2Fcontract-runtime/topic/NEP-364. There are some interesting discussions about this PR.

akhi3030 avatar Jul 12 '22 14:07 akhi3030

@akhi3030 I've removed all non signature verification related code. So that hashing related functions are moved onto another NEP (will add it shortly).

I'm not sure about https://github.com/near/nearcore/pull/7165#issuecomment-1181830146 What's needed from my side?

blasrodri avatar Jul 13 '22 08:07 blasrodri

@akhi3030 I've removed all non signature verification related code. So that hashing related functions are moved onto another NEP (will add it shortly).

I'm not sure about #7165 (comment) What's needed from my side?

Thanks @blasrodri! This comment in the zulip discussion was particularly interesting: https://near.zulipchat.com/#narrow/stream/295306-pagoda.2Fcontract-runtime/topic/NEP-364/near/289315918. We have to figure out how to make sure that all the validations are deterministic. I recommend having a chat with @evgenykuzyakov to further understand the concerns.

akhi3030 avatar Jul 13 '22 09:07 akhi3030

@akhi3030 could you review this PR again?

bowenwang1996 avatar Aug 04 '22 23:08 bowenwang1996

finally, got green :)

blasrodri avatar Aug 09 '22 19:08 blasrodri

@jakmeier I just ran it on another machine (M1) using

export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=1;
export CARGO_PROFILE_RELEASE_LTO=fat;
     Running `target/release/runtime-params-estimator --accounts-num 20000 --additional-accounts-num 20000 --iters 5 --warmup-iters 1 --metric time --costs Ed25519VerifyBase,Ed25519VerifyByte`
[elapsed 00:00:03 remaining 00:00:00] Writing into storage โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ   20000/20000  
Ed25519VerifyBase                                   73_232_640_700 gas [                 73.233ยตs]           (computed in 17.18s) 
Ed25519VerifyByte                                      754_106_187 gas [                    754ns]           (computed in 9.14s) 

blasrodri avatar Aug 22 '22 09:08 blasrodri

@jakmeier I just ran it on another machine (M1) using

export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=1;
export CARGO_PROFILE_RELEASE_LTO=fat;
     Running `target/release/runtime-params-estimator --accounts-num 20000 --additional-accounts-num 20000 --iters 5 --warmup-iters 1 --metric time --costs Ed25519VerifyBase,Ed25519VerifyByte`
[elapsed 00:00:03 remaining 00:00:00] Writing into storage โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ   20000/20000  
Ed25519VerifyBase                                   73_232_640_700 gas [                 73.233ยตs]           (computed in 17.18s) 
Ed25519VerifyByte                                      754_106_187 gas [                    754ns]           (computed in 9.14s) 

Yeah that looks about right. Good to see the uncertainty warning is gone now!

But let's keep the numbers that you already had. The M1 platform is not suited well for our estimations for a number of reasons. And we will need to do fine-tuning anyway before stabilising this.

I think the estimation part is looking good now for this PR.

@matklad do you want to take another more holistic look at the code before we merge it?

jakmeier avatar Aug 22 '22 12:08 jakmeier

@matklad do you mind reviewing this PR again?

bowenwang1996 avatar Aug 26 '22 04:08 bowenwang1996

@blasrodri sorry, I forgot auto-merge label last time, could you update this, so that we get this in?

matklad avatar Sep 06 '22 12:09 matklad

Created a tracking issue for a bunch of unresolved questions we have here: https://github.com/near/nearcore/issues/7567

matklad avatar Sep 06 '22 16:09 matklad