React Native: Solutions for slow crypto-operations (`Wallet.createRandom()`)

[mrousavy]

A lot of issues have been opened about React Native being slow. For example, creating a wallet takes 33 seconds on an iPhone 11 Pro - I don't even want to find out how slow that is on an older Android device.

This code:

import 'react-native-get-random-values'
import '@ethersproject/shims'
import { ethers } from 'ethers'


  const now = performance.now()
  Logger.log('💰 Creating new Wallet...')
  const wallet = ethers.Wallet.createRandom()
  const end = performance.now()
  Logger.log(
    `💰 New wallet created! Took ${end - now}ms, Phrase: ${wallet.mnemonic.phrase}`,
  )

Takes half a minute:

[16:49:19.253]	💰 Creating new Wallet...
[16:49:52.812]	💰 New wallet created! Took 33559.440958321095ms, Phrase: ...

See https://github.com/facebook/hermes/issues/626

I'm wondering if there is any workaround to make this code faster, as it's definitely not a solution to let the user wait for 5 minutes until he can use the app.

I can create a library for the crypto polyfills that uses a pure C/C++ implementation accessed synchronously through JSI for maximum performance, but I'd need help to find out what algorithms are slow and where I can swap them.

[mrousavy]

For example, for atob and btoa this JSI library by @craftzdog is 4x faster than base64-js: https://github.com/craftzdog/react-native-quick-base64

[mrousavy]

~~I'd love to make the pbkdf2 function asynchronous (run the algorithm on a separate Thread, return a Promise) to not block the entire JS interaction (pressing buttons, updating loading percentage, etc.)~~

nvm it only takes 1ms if written in C++, no need to make it async.

[mrousavy]

I did it.

@zemse @mirceanis @ricmoo how do I swap the pbkdf2 function?

The one from this library takes 14 seconds:

ether's pbkdf2 took 14279.91887497902ms

and my pure C++ JSI implementation takes 0.001 second:

C++ pbkdf2 took 1.4742500185966492ms

..so it's almost 10.000x faster.

I'd love to make that open source if there's a way to swap the implementations.

[mrousavy]

So I patched @ethersproject/pbkdf2/lib/browser-pbkdf2.js to use my custom C++ implementation instead of the JS one, and here's the results:

Before

💰 Importing wallet from phrase: ....
💰 Successfully imported wallet in 45114.42308330536ms!

After

💰 Importing wallet from phrase: ....
💰 Successfully imported wallet in 1120.2051666378975ms!

...so I brought it down from 45 seconds to 1 second, a 45x performance improvement.

My patch

diff --git a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
index 45c6f03..41e4bf6 100644
--- a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
+++ b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
@@ -1,47 +1,10 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.pbkdf2 = void 0;
-var bytes_1 = require("@ethersproject/bytes");
-var sha2_1 = require("@ethersproject/sha2");
+
 function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) {
-    password = (0, bytes_1.arrayify)(password);
-    salt = (0, bytes_1.arrayify)(salt);
-    var hLen;
-    var l = 1;
-    var DK = new Uint8Array(keylen);
-    var block1 = new Uint8Array(salt.length + 4);
-    block1.set(salt);
-    //salt.copy(block1, 0, 0, salt.length)
-    var r;
-    var T;
-    for (var i = 1; i <= l; i++) {
-        //block1.writeUInt32BE(i, salt.length)
-        block1[salt.length] = (i >> 24) & 0xff;
-        block1[salt.length + 1] = (i >> 16) & 0xff;
-        block1[salt.length + 2] = (i >> 8) & 0xff;
-        block1[salt.length + 3] = i & 0xff;
-        //let U = createHmac(password).update(block1).digest();
-        var U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, block1));
-        if (!hLen) {
-            hLen = U.length;
-            T = new Uint8Array(hLen);
-            l = Math.ceil(keylen / hLen);
-            r = keylen - (l - 1) * hLen;
-        }
-        //U.copy(T, 0, 0, hLen)
-        T.set(U);
-        for (var j = 1; j < iterations; j++) {
-            //U = createHmac(password).update(U).digest();
-            U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, U));
-            for (var k = 0; k < hLen; k++)
-                T[k] ^= U[k];
-        }
-        var destPos = (i - 1) * hLen;
-        var len = (i === l ? r : hLen);
-        //T.copy(DK, destPos, 0, len)
-        DK.set((0, bytes_1.arrayify)(T).slice(0, len), destPos);
-    }
-    return (0, bytes_1.hexlify)(DK);
+    // uses my C++ implementation.
+    return global.pbkdf2(password.buffer, salt.buffer, iterations, keylen, hashAlgorithm)
 }
 exports.pbkdf2 = pbkdf2;
 //# sourceMappingURL=browser-pbkdf2.js.map

I still wonder why it takes a full second to import the wallet from the passphrase, I am even using the native C++ atob, btoa and Buffer implementations, will debug further - but at least now the app is useable.

[zemse]

Can you try using resolutions in package.json to override the pbkdf2 subpackage? (I'm currently afk otherwise would include an example)

[mrousavy]

I'll try to play with resolutions too, right now I have used patch-package to locally patch it - quick and dirty but it works.

Since getting the wallet still takes more than a second on a relatively new iPhone, I'm not done with optimizing yet. Unfortunately I'm having trouble getting the flamegraph profiler working in React Native, do you maybe know off the top of your head which other functions are causing those performance issues that are worth looking into? I can also implement those in C++

[ricmoo]

Just as an FYI, I've been adding an option to v6 to register custom implementations for certain crypto methods, so this should be easier to swap out in v6.

[mrousavy]

Do you have an ETA for v6?

[sandys]

@ricmoo this is interesting for us too. actually we have been tracking this using another issue - https://github.com/ethers-io/ethers.js/issues/1503

im betting both will get fixed by the new swapouts.

[ricmoo]

I'm working on v6 as fast as I can and hope to get a beta out before the end of January. There are a few priority changes needed for another minor bump though, which might bump that a bit.

[Cakesoft-Sameer]

@mrousavy this patch seems to be the end to my quest.But still unable to get it working throwing "undefined" while using it.Could you shed some light on the usage

[maykonmichel]

There is something that we can do to help you release v6?

[andra961]

@mrousavy this patch seems to be the end to my quest.But still unable to get it working throwing "undefined" while using it.Could you shed some light on the usage

I know nothing about jsi but i think you still need to have the module itself installed in the global object somehow. @mrousavy can you please share the pbkdf2 c++ module too so we can try it out?(if i missed it or i misunderstood something please correct me)

[Thinhnguyen185]

@mrousavy hmm, patch seem don't used, global always "undefined", peformace when run createRandom() on andoird very bad

[mrousavy]

Well obviously you still need the native function, otherwise the patch won't work. That's the whole idea, moving it to native. 😅

The code is closed source for a client of mine (at Margelo), but we will release something that speeds up this operation (and all other crypto operations) by magnitudes soon.

@ricmoo I think I mentioned this before, but we're essentially bringing crypto to React Native with C++ implementations instead of JS browser-polyfills, so this will be a huge performance improvement 🔥 I'll DM you on Twitter once we are ready to test the integration with ethers

[linxianxi]

@ricmoo When will V6 be released? My application is about to go into production, so I may have to switch to web3.js

[ricmoo]

There is a v6 beta available now, but I haven’t got any meaningful documentation together yet.

If the new Wallet(utils.randomBytes(32)) method doesn’t work for you (which should be instant, even on react-native), you could probably trivially just implement HDNode using native libraries, or there are probably already existing libraries for that part designed for RN.

But if you go the Web3.js route, I’d be curious how easy it is to switch between the two.

[linxianxi]

There is a v6 beta available now, but I haven’t got any meaningful documentation together yet.

If the new Wallet(utils.randomBytes(32)) method doesn’t work for you (which should be instant, even on react-native), you could probably trivially just implement HDNode using native libraries, or there are probably already existing libraries for that part designed for RN.

But if you go the Web3.js route, I’d be curious how easy it is to switch between the two.

new Wallet(utils.randomBytes(32)) does not generate mnemonic words.web3.js integration with react-native is more difficult and has no documentation. If you can tell us how to use it in V6 here.Thanks.

[sandys]

Same here. I would prefer to use v6 beta and put my app in production. Rather than switch between libraries.

Anyway u could give us a complete example on how to use v6 ?

On Thu, 14 Apr, 2022, 17:35 linxianxi, @.***> wrote:

There is a v6 beta available now, but I haven’t got any meaningful documentation together yet.

If the new Wallet(utils.randomBytes(32)) method doesn’t work for you (which should be instant, even on react-native), you could probably trivially just implement HDNode using native libraries, or there are probably already existing libraries for that part designed for RN.

But if you go the Web3.js route, I’d be curious how easy it is to switch between the two.

new Wallet(utils.randomBytes(32)) does not generate mnemonic words.web3.js integration with react-native is more difficult and has no documentation. If you can tell us how to use it in V6 here.Thanks.

— Reply to this email directly, view it on GitHub https://github.com/ethers-io/ethers.js/issues/2250#issuecomment-1099046177, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASYUZR5PUHRK34JGCTYJDVE7YIVANCNFSM5HA54GVA . You are receiving this because you commented.Message ID: <ethers-io/ethers .@.***>

[zemse]

Anyway u could give us a complete example on how to use v6 ?

Until the official docs are up here's an example to use custom pbkdf2 implementation.

IMP: This code example needs a fast pbkdf2 implementation in RN. Just copy pasting this below snippet would NOT WORK.

import { Wallet } from "ethers";
import { pbkdf2 } from "@ethersproject/crypto";

// IMP: this code WON'T WORK by simple copy paste in React Native, you need to replace the following with a
// working implementation of pbkdf2 in RN.
import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

/**
 * @param {*} password Uint8Array
 * @param {*} salt Uint8Array
 * @param {*} iterations number
 * @param {*} keylen number
 * @param {*} algo "sha256" | "sha512"
 * @returns BytesLike
 */
function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

// registers custom implementation for pbkdf2
pbkdf2.register(customPbkdf2);

console.log("creating wallet");
const wallet = Wallet.createRandom();
console.log(wallet.address);

I would prefer to use v6 beta and put my app in production.

Note that v6 is in very early beta as mentioned by ricmoo.

[linxianxi]

Anyway u could give us a complete example on how to use v6 ?

Until the official docs are up here's an example to use custom p

import { Wallet } from "ethers";
import { pbkdf2 } from "@ethersproject/crypto";

import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

/**
 * @param {*} password Uint8Array
 * @param {*} salt Uint8Array
 * @param {*} iterations number
 * @param {*} keylen number
 * @param {*} algo "sha256" | "sha512"
 * @returns BytesLike
 */
function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

// registers custom implementation for pbkdf2
pbkdf2.register(customPbkdf2);

console.log("creating wallet");
const wallet = Wallet.createRandom();
console.log(wallet.address);

I would prefer to use v6 beta and put my app in production.

Note that v6 is in very early beta as mentioned by ricmoo.

This will report an error,did you try it on RN?

[zemse]

Which line does it error on?

import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

You need to replace the above code with a pbkdf2 method that works nicely in RN.

[andra961]

Which line does it error on?

import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

You need to replace the above code with a pbkdf2 method that works nicely in RN.

Pardon me where can i get the v6 code to try this out?

[ricmoo]

@andra961 it’s the v6-beta branch of this repo, or npm install ethers@beta. There is no documentation yet.

[zemse]

@mrousavy Do you have the pure C++ JSI implementation mentioned in the reply above available for public? The example here is for swapping the pbkdf2 ethers would use.

[andra961]

@mrousavy Do you have the pure C++ JSI implementation mentioned in the reply above available for public? The example here is for swapping the pbkdf2 ethers would use.

He already said it's closed source

[Thinhnguyen185]

I find solution for slow when create random wallet ` import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; };

const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic);

return { wallet: new ethers.Wallet(privateKey), mnemonic, }; }; `

[andra961]

I find solution for slow when create random wallet ` import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; };

const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic);

return { wallet: new ethers.Wallet(privateKey), mnemonic, }; }; `

How can you make bip39 work on react native?

[Thinhnguyen185]

I find solution for slow when create random wallet import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; }; const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic); return { wallet: new ethers.Wallet(privateKey), mnemonic, }; };

How can you make bip39 work on react native?

https://www.npmjs.com/package/bip39 this work on RN

[andra961]

I find solution for slow when create random wallet import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; }; const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic); return { wallet: new ethers.Wallet(privateKey), mnemonic, }; };

How can you make bip39 work on react native?

https://www.npmjs.com/package/bip39 this work on RN

I tried and it didn't. Pretty sure it requires shims to work. Are you using something like rn-nodeify?