cryptography
cryptography copied to clipboard
dint-dev
Pbkdf2 key derivation taking too long
I implemented the Pbkdf2 key derivation function in my code, but its execution is taking way too long. I followed the documentation and came up with:
final pbkdf2 = Pbkdf2(
macAlgorithm: Hmac.sha256(),
iterations: 100000,
bits: 128,
);
final newSecretKey = await pbkdf2.deriveKey(
secretKey: secretKey,
nonce: nonce,
);
final newSecretKeyBytes = await newSecretKey.extractBytes();
I tested it with a 32 bit nonce on the android emulator and it took about 2 seconds to derive a 5 lettered String. Did the same thing in Python with exactly the same Algorithm, iterations etc. and this took only 0.2 seconds, which is actually average for the calculation of Pbkdf2. I also tested Hkdf on the emulator which was quite fast, but i have no information on how it works and how it compares to Pbkdf2, so i cant judge about that result.
Turned out that the problem was in my code. I set an absurd high iteration number of 100.000, which slowed my code enormously. According to this article, it is definetely enough for most purpouses to go with the 45.000 iterations. What wonders me though is that the python example executed so fast (Python is rather slow). Maybe in the package i use (cryptography) they have some kind of threshold to save processor power.
Did you test your code only in debug mode?
As mentioned on flutter.dev: "Application performance can be janky in debug mode. Measure performance in profile mode on an actual device."
I'm planning to use Pbkdf2 in my application too since Argon2 isn't supported yet. I did some initial experiments and so far I can confirm that it takes approximately 2 seconds in debug mode with the suggested parameters. For my use case this isn't much of a problem though. I'll try to measure the performance in profile or release mode and share my results.
I tested it in debug mode but I did an APK release additionally, which takes also too long on my mobile phone.
I'm waiting for Argon2id as well, which I found a documentation of. Unexpectedly, it only returns an UnimplementedError. Not the best way to show they didn't implement it yet.
If you (@maxfornacon) also report slow encryption in release mode, i would reopen this issue again.
@PaulanerAlex I can confirm this. I'm currently double checking that it is not an initialization error, that the output fits. But so far I see this: The same algorithm parameters showed an unexplainable abysmal performance for cryptography. Where crytography takes ~40 seconds for PBKDF2 with 200k iteration - in performance mode on a real device - webcrypto takes ~0.2s. And checking against ruby/openssl 0.2s is the expected performance.
Additionally, that's about half as fast as cryptography has been before version 2.0 on the same device, likely slower. Something seems to be completely wrong with the PBKDF2 algorithm, or the parameters are interpreted completely differently than what one would expect, as you had seen with the Python comparison.
Turned out that the problem was in my code. I set an absurd high iteration number of 100.000, which slowed my code enormously.
I might be wrong, but from my research 100.000 is not too high, it's even rather low. Note how 100k was the default in 2016 in the article you linked, 5 years ago. You can be almost certain that there is no cap lower than that on the python implementation. Your python comparison then seems to confirm that something is wrong here, and my ruby script does as well.
I might be wrong, but from my research 100.000 is not too high, it's even rather low. Note how 100k was the default in 2016 in the article you linked, 5 years ago. You can be almost certain that there is no cap lower than that on the python implementation. Your python comparison then seems to confirm that something is wrong here, and my ruby script does as well.
Okay, @maltemedocs that seems plausible. I will open the issue again.
Im having the same issue. Even with 45000 iterations it takes about 2.1 seconds to create the key
I have tested two emulators and one actual device (Xiaomi Mi 8). 45000 iterators Pixel 2 API 30: 1105 ms iPhone 8: 834 ms Xiaomi Mi 8: 1093 ms 100000 iterators Pixel 2 API 30: 2126 ms iPhone 8: 1546 ms Xiaomi Mi 8: 2193 ms I also tested alternative packages, but this package is the fastest. 100000 iterators Nodejs performance: 252 ms
Performance of the pure Dart version of PBKDF2 suffered from async calls and hash state allocations on each iteration. These issues were eliminated by version 2.4.0 - at least for SHA256, Blake2s, and Blake2b.
After fixing the earlier performance issues, the pure Dart performance is still 10x-20x worse than optimized C implementations in browsers. The problem is that pure Dart SHA256/SHA512 hash functions are about 10x times worse and can't be optimized much more. A benchmark using 100k iterations and 128 bit output:
DartPbkdf2(
macAlgorithm: DartHmac.sha256(),
iterations: 100000,
bits: 128,
).deriveKey(...): 4 op / second
DartPbkdf2(
macAlgorithm: DartHmac.sha512(),
iterations: 100000,
bits: 128,
).deriveKey(...): 2 op / second
BrowserPbkdf2(
macAlgorithm: BrowserHmac.sha256(),
iterations: 100000,
bits: 128,
).deriveKey(...): 94 op / second
BrowserPbkdf2(
macAlgorithm: BrowserHmac.sha512(),
iterations: 100000,
bits: 128,
).deriveKey(...): 19 op / second
@terrier989 I can confirm that it is a lot better now. I'm not sure where I tested the 100k iterations before that took 40 seconds, but according to old documentation I found, on an LG G5 100k iterations took 51 seconds with an old version of cryptography. With 2.5.0 they take 2s now, 200k take 5s (inside an async compute task) -> that's not ideal, but maybe even already usable. At the very least not a complete blocker anymore.
Thank you for the improvement!
Whenever possible, I recommend using Argon2id password hashing algorithm, which is now fully implemented by this package.
I close this ticket because significant further improvements to PBKDF2 performance are unlikely.