BIP38 decrypt may have a browser dependency (Safari 6.x issue)

[cantonbecker]

I've been doing some testing with BIP38 decryption at bitaddress.org and ran into a funny issue when encrypting/decrypting wallets using the passphrase "घोडा स्टेपल" (no quotes) -- which is what you get when you use google translate to translate "horse staple" (no quotes) into Nepali.

My own BIP38 generator is, weirdly enough, giving me two different BIP38 encodings for the same wallet, whose details are: public key 1ABCDF5v4oaodTPYnKfYvkfwuoa8PJkjMC WIF key 5J4pcwBDPwPY1cdNqxTdmZWr7yCK8rXi9avFvezgYbmoatJpKGn

If I use Safari to encrypt with घोडा स्टेपल, I get 6PRNXA7M57uqSYXX2TXHkfNJEVMiWarkPkqUv3AsZa5r41u3VpXHLkUD9q

If I use Chrome / Firefox / IE and encrypt with घोडा स्टेपल, I get 6PRNXA7M4qEppBJCHM2SEizfna7XTomzXwdCBrEG6Mjo3nU6iziS6vWWXA

I'm not sure if this is my own bug, or something native to the BIP38 implementation I'm borrowing from bitaddress.org. Hard to test because I don't think bitaddress.org will let me BIP38 encrypt my own vanity address or brain wallet.

But here's something you can replicate / experience. I've been using bitaddress.org to check the validity of my BIP38 wallets. What I do is I fire up bitaddress.org, open up the "wallet details" tab, and use this to decrypt my "6P..." keys. And I'm getting browser-dependent results:

*** Using the 'wallet details' tab on bitaddress.org to decrypt the BIP38 key, if using safari, the safari-generated key (_9q) works, and the other-browsers key (_XA) fails to decrypt. If using chrome/FF/IE, the reverse is true. ***

I'm experiencing the same browser-dependent decrypting success/failure using the "decrypt private key" function at bit2factor.org.

Any ideas on why bitaddress.org is unable to decrypt my Nepali-encoded private keys unless I'm using the same browser I used to generate those keys?

[cantonbecker]

PS I realize it's silly (maybe even obnoxious) for me to be doing tests between two completely different projects -- my project (bitcoinpaperwallet.com) and bitaddress.org. As such I did some playing around and succeeded in generating a wallet using bitaddress.org that can only be recovered if you use the same browser to decrypt it.

Here's what I did:

1. Using bitaddress.org and OS X Safari I set my BIP38 passphrase to "घोडा स्टेपल" (without the quotes) which is the google translates response for translating "horse staple" to Nepali.

2. Generated one BIP38 wallet. See http://cl.ly/image/0m0S1N1p2Y3P

3. Took the resulting private key 6PfUfMYT9PZpJ5q5NCosyqW3pMxwpu24G6mqCVbS41ND7aWctqeqBqz7NJ and fed it into the "Wallet Details" function at bitaddress.org along with the "घोडा स्टेपल" Nepali passphrase.

Result: Decrypts fine. http://cl.ly/image/3p2X332x370X

4. Opened up firefox. Opened up Wallet Details and tried to decrypt the same key with the same password.

Result: "Incorrect passphrase for this encrypted private key".

I haven't confirmed this myself yet, but mannkind (bit2factor.org) was unable to successfully decode this wallet at bitaddress.org using his own Safari. (Furthermore, his own bit2factor.org service appears to have some browser-dependent decryption ability.) If true, this means you may even need the same version of the browser, which would be pretty alarming as it significantly reduces the ease of retrieval.

What's worrisome about this possible issue is that someone generating a BIP38 wallet might very reasonably test their ability to decrypt the wallet before loading the wallet with bitcoins. But they'd almost certainly be using the exact same browser, which would circumvent this possible issue.

NB: I have only been testing the matrix of OS X Safari 6.x, OS X Chrome, OS X Firefox, Windows IE9, and Ubuntu Firefox. I haven't done any more extensive tests with Windows or especially Opera. If someone is feeling ambitious, a UTF passphrase test suite should probably be run testing encryption & decryption round trips using a god-awful extended charset passphrase like "घोडा स्टेपलʏpsilɔn⡍⠜⠇⠑⠹ŠŸžΚαλημέンニチอขัน".

At the moment I'm thinking the problem here is something to do with the BIP38 code where depending on the browser you're using (and with Safari in particular) some extended charsets are interpreted differently -- whether encrypting or decrypting -- which is why you are obligated to use the same browser to decrypt as you used to encrypt.

[cantonbecker]

NB: Back on the bitcoinpaperwallet.com, artiomchi did some testing with various versions of safari and may have narrowed things down significantly. It might be that BIP38 is encrypted (and decrypted) differently specifically when using Safari 6 as compared with 5 or 7. See:

https://github.com/cantonbecker/bitcoinpaperwallet/pull/6#issuecomment-31838184

[pointbiz]

In Safari 6 do all the async unit tests pass? Query string: ?asyncunittests=true

[cantonbecker]

Unfortunately, at least using safari 6.05, no.

See: http://cl.ly/image/2L0P1y1p0U3r

running 8 tests named testDecryptBip38 running 4 tests named testBip38Encrypt running 2 tests named cycleBip38 running 5 tests named testBip38Intermediate pass testDecryptBip38 #0 pass testDecryptBip38 #1 pass testDecryptBip38 #2 pass testDecryptBip38 #3 pass testDecryptBip38 #4 fail testDecryptBip38 #5, error: Incorrect passphrase for this encrypted private key. pass testDecryptBip38 #6 fail testDecryptBip38 #7, error: Incorrect passphrase for this encrypted private key. pass testBip38Encrypt #0 pass testBip38Encrypt #1 pass testBip38Encrypt #2 pass testBip38Encrypt #3 pass cycleBip38 test #0 pass cycleBip38 test #1 pass testBip38Intermediate #0 pass testBip38Intermediate #1 fail testBip38Intermediate #2, error: Incorrect passphrase for this encrypted private key. fail testBip38Intermediate #3, error: Incorrect passphrase for this encrypted private key. fail testBip38Intermediate #4, error: Incorrect passphrase for this encrypted private key. running of asynchronous unit tests complete!

[pointbiz]

Well that is somewhat good news. Better that the unit test also fails. Are the failures intermittent?

Unfortunately, those unit tests take so long to run that we can't use them to detect there is a problem for average users.

[cantonbecker]

PS: I should mention 3 things we've figured out recently:

1. Has nothing to do with UTF8 / extended charset. Problem occurs with very ordinary passphrases like "my passphrase".

2. May not occur on Safari 5, 7, or even 6.1. Might be limited to 6.05 and possibly 6.0 (the typical OS 10.7.x browser) which I estimate makes up around 2.5% of paper wallet users.

3. It's intermittent. Notice the results from running the asyncunittests a second time fail in different spots:

running 8 tests named testDecryptBip38 running 4 tests named testBip38Encrypt running 2 tests named cycleBip38 running 5 tests named testBip38Intermediate pass testDecryptBip38 #0 pass testDecryptBip38 #1 fail testDecryptBip38 #2, error: Incorrect passphrase for this encrypted private key. pass testDecryptBip38 #3 pass testDecryptBip38 #4 pass testDecryptBip38 #5 pass testDecryptBip38 #6 pass testDecryptBip38 #7 pass testBip38Encrypt #0 pass testBip38Encrypt #1 fail testBip38Encrypt #2 expected 6PYNKZ1EAgYgmQfmNVamxyXVWHzK5s6DGhwP4J5o44cvXdoY7sRzhtpUeo received 6PYNKZ1EBJQFCNyr5QsjAa7GtL3e5Z4tZbbtoA3M26Sp6Ljmot8FAwofwG pass testBip38Encrypt #3 pass cycleBip38 test #0 pass cycleBip38 test #1 fail testBip38Intermediate #0, error: Incorrect passphrase for this encrypted private key. pass testBip38Intermediate #1 fail testBip38Intermediate #2, error: Incorrect passphrase for this encrypted private key. pass testBip38Intermediate #3 pass testBip38Intermediate #4 running of asynchronous unit tests complete!

[artiomchi]

After spending a bunch of time (way more than I should have today), I have traced the issue to Crypto_Scrypt library so far. I'm not sure if it's this library, or something that it depends on.

Here's a simple example to demonstrate the issue: http://jsutftest.azurewebsites.net/testCrypto.html ( page blocking, wait for it to process first )

[artiomchi]

Ok, got some further progress on it. I've narrowed it down to the scrypt library, and managed to replicate it with minimal code.

Since no other library is required to replicate the test, I've opened an issue in the scrypt-js project, where we can follow-up, and hopefully find a fix.

cheongwy/node-scrypt-js#2

[scintill]

I am also seeing something nondeterministic. Starting with @artiomchi's minimal testcase, I am dumping some intermediate state and comparing between Chrome and Safari 6.0.5. I have just observed several times in a row, that the first run of the script in a new window gives incorrect intermediate state, while refreshing gives correct answers (or at least the bytes I am looking at are correct.) I was closing windows because sometimes the long calculations seem to have hung... not sure if just impatient or what. Refreshing does not make @artiomchi's unaltered testcase come out correctly.

There shouldn't be any way for JS code to be causing this, right? Even if you're using globals, they should be refreshed with the page? I can't think of any standard caching effects that should be involved here. If I'm correct, there is something wrong with Safari 6.0.5 (and others?) and I'm not sure we can work around it.

I don't know if the maintainer of cheongwy/node-scrypt-js will help, because the project is built for Node where presumably this issue does not arise. I'm the one who repackaged that scrypt code and added web workers, for bitaddress.org.

[artiomchi]

That's indeed the case, @scintill. Basically, I think Safari (not sure if it's limited to v6 or not) has some strange "optimisation" for javascript, which is causing a bunch of issues. For example: A bunch of times I added some console.log statements to log states, and figure out what was going wrong - and the values in the console were different to the values that the variables actually had in code.

For example:

B = [large array];
console.log(B);
B = [values changed in array];

The console output would actually sometimes show the new state of the array when I triggered that in Safari 6. Took me a while to figure this out =/

Anyway, regarding the minimal testcase, you're absolutely right to note that if you re-run the test case it will give correct answers. But if you refresh it - it will give incorrect results.

Try something else. Open the page, open the developer console, and then try refreshing the page. If I understood this correctly, having the developer console open slows down the page just enough to avoid whatever issue we are having, and gives the correct results during the first go. At least it did that to me half the time. The other half the time the browser hung in the VM (it really didn't like the dev console) :)

Could you check that out please, @scintill, and report back?

I understand that the original project may not be updated, but I created the issue there, since that's where the bug is, and so far the problem is tracked among 4 different projects in GitHub, I thought it would be better to point it to the origin.

Great job repackaging the code for the web workers, by the way! Very well done! :)

[pointbiz]

I closed https://github.com/pointbiz/bitaddress.org/issues/48 as a duplicate and updated the subject of this issue to note it is a Safari 6.x only issue

[scintill]

Aha, I believe we are experiencing the bug reported here, a WebKit JIT bug present in several versions and flavors of Safari.

This explains the symptoms, such as it running much faster when it gives the wrong answer (JIT speeds up execution but does it incorrectly); and working, but slowly, when the dev console is enabled (the linked WebKit bug report says JIT is disabled when the console is open.)

This is quite alarming and frustrating that such a severe bug made it into relatively wide deployment, ugh. Even if we appear to work around it, I am a bit reluctant to let people risk money in a broken JS engine. On the other hand, I'm not sure we can even reliably detect every browser that is affected.

I've had about 6 successful runs, with devconsole closed, with a modified testcase here. It's not a thorough test, but I think it's a good indicator. I am disabling JIT by wrapping every function body in try/catch as advised at StackOverflow. Unfortunately this is much slower, even on Chrome (multiple "This script appears to be hung" popups.) I suspect speed could be improved by de-JIT'ing fewer functions and possibly rewriting things to work around the JIT bugs. But again, I am reluctant to be conservative at all given what's at stake.

Let's see if I do this right. Cc other bugs: cantonbecker/bitcoinpaperwallet#6, mannkind/bit2factor.org#2, cheongwy/node-scrypt-js#2.

[artiomchi]

Great job pointing it out!

By the way, it's enough putting try/catch blocks only around the salsa20_8() method.. It's still considerably slower (10sec vs 60sec on my machine), but it ends up working on Safari 6 as well...

In fact. After trying out different things, I realised that if I comment out the following line in salsa20_8(), the results will be the same in Chrome and Safari 6

B32[i] |= (B[i * 4 + 3] & 0xff) << 24;

Anyway, working on after that, I believe I managed to get my testcase working by calling salsa20_8() ahead of time with a dummy array. Since only the first execution yielded bad results, it seems like this call may have gotten it working.. Could you guys test it on your side, since my VM is incredibly slow, and it would be good to have multiple test inputs...

I've updated my gist, and the hosted version above (http://testcrypto.azurewebsites.net/) with the potential "fix"

[cantonbecker]

Hi Joey,

"I am disabling JIT by wrapping every function body in try/catch as advised at StackOverflow."

Nice research you did, and this sounds like a fine work-around. I just wanted to write a reminder to everyone impacted that before implementing any kind of a fix on a site like bitaddress.org, there should be some plan on how to support wallets already made using the broken Safari.

Right now those users are still able to decrypt their BIP38 wallets by using the same version of Safari as they used to create it. If this bug is worked-around, then those same users would not only have to use the correct version of safari, but also hunt down a pre-2014 release of bitaddress.org to do the decrypting. Ideally, a 2014 release of bitaddress should have some facility for attempting to decrypt using the old bug-exposing code as well as the new bug-work-arounding code.

[artiomchi]

One disappointing thing to note is that it's not a bug in the javascript code, but instead a bug in the browser's execution of the code. That is to say - the browser is executing the code incorrectly in some way (I think the execution order of some statements bay be skewed).

As such, the only way to get this to work is to find which order did Safari actually execute the statements, and create a "dummy" method that replicates that behaviour.. That would require a whole bunch of changing/breaking the code, executing it on a proper browser like Chrome, and hope you get the "correct" invalid result.. Eek!

Although if I'm right, and the issue is indeed with salsa20_8() that narrows things down a lot!

[scintill]

Artiom, good work on narrowing that down! I verified your finding, that a dummy call to salsa20_8() beforehand makes the whole scrypt work (or at least the first 50 bytes we are comparing.) I also made some small modifications, to only do the dummy call once, and to re-try the calculation without refreshing the page -- these were successful.

I also verified that taking out the << 24 line seems to make consistent (but incorrect) results between Chrome and Safari, without the Safari dummy call. Good work on finding that too. I've tried tweaking the surrounding lines to see if I could learn anything else, but little luck yet. I did find that "unrolling" that 0-16 "B32" loop (writing out B32[0] = (B[0]...) caused a different incorrect result in Safari, but I am not sure if that helps.

I am a little hesitant to say this dummy call is a "fix" since we don't fully understand the issue, but it's a great start, seems to be working, and may end up being good enough.

I would also like to replicate the broken Safari calculation in other browsers, but it seems like the only options are randomly twisting the code until Chrome gives the same "broken" result, or trying to reverse-engineer the JIT to find out what the code it creates is really doing. Neither is going to be quick or pleasant...

I am wondering if parts of the smix -> blockmix_salsa8 -> salsa20_8 call chain is being inlined since blockmix_salsa8 is called in long loops in smix (N = 16384, twice.) Maybe calling salsa20_8 somewhere else causes the JIT to make a different decision here. I don't know if this helps us, though.

[cantonbecker]

Just wondering if this issue is still being tracked for bitaddress.org? At the very least it might be worth having an alert pop up for Safari 6.05 users saying "don't make BIP32 wallets using this browser!"

Meanwhile, I'm rolling out BIP32 for bitcoinpaperwallet.com in the next few days with the work-around implemented by artiomchi. It seems to make Safari 6.05 reliable for proper address encoding/decoding.

[abkaya]

This may be too late to reopen the subject, but I have this particular issue. I saved a couple bitcoins over 4 wallets generated and BIP-38 encrypted in safari 6 at the time.

Apple doesn't offer a way to revert to safari 6.05. Would you have any suggestions for me? It would be saddening to see it lost, because of this.

Thank you for the work you guys put into it nonetheless. At least now, I've got an idea as to why the passphrase was 'wrong'.

[pointbiz]

@abkaya I would not consider your bitcoins lost. I need to review the issue but I believe Safari 6 was behaving incorrectly in a deterministic way. Meaning myself or someone on this thread like @scintill or @cantonbecker or @artiomchi could create a branch that mimics the Safari 6 behavior. So users like yourself could decrypt their keys.

So keep watching this issue.

[abkaya]

@pointbiz Thank you very much, I will definitely keep an eye out.

[artiomchi]

The problem is - I'm not really sure where Safari messed up right there - it's definitely an issue, but replicating it on any other browser is a whole different project.

@abkaya - in your case, it's not all lost, actually. You should be able to use BrowserStack to create a vm with this version of Safari (that's how I was able to test it), to get the private key decrypted (although I'd suggest moving them to a new wallet straight away after that)

Good luck!

[artiomchi]

I should probably note that Safari 6.05 was indeed behaving incorrectly (hence you can't decode your wallet now), but in a consistent manner. So to decode your wallet, you'd just need to replicate the same environment. Last time I was working on this issue (looking at the comments) we seem to have found a "workaround" to make it work in the Safari browser as well. I'm not sure if this is live now, so if the latest copy of the website can't decode it, you can just go back in history, and use an older version of bitaddress.org/bitcoinpaperwallet.com that didn't have the patch implemented..

If that doesn't work, ping us a message, and I'm sure we'll find a solution one way or another :)

[scintill]

Yes, I also think it's deterministic, the only redeeming factor in this. I would second the BrowserStack suggestion as I used it for testing back then too, but I don't see Safari 6.0.5 on their supported browser list anymore. :( I can't remember if we had any data on what Safari 6.0 did, but it's listed under Mac OS X Lion (64-bit) so we might try that.

You might try finding a download of that specific version on some site like this. I would be extremely cautious with this, though! Verify a code signature from Apple if possible (this might help), and install on a fresh disposable machine (maybe a virtual machine) to limit the damage from any malware. This may be above @abkaya's skills to do securely -- if so, I'd be willing to test it so we can see if it's worth pursuing further.

@abkaya, similar to how artiomchi mentioned, I would recommend trying to reproduce the original testcase (first post) first, rather than repeatedly trying your real keys and passphrase into BrowserStack or a questionable download.

I have a few old notes about trying to make a corresponding WebKit build, which might have the same bug -- unfortunately I don't have notes or memory about whether I had success. I'll look into it again. This could help developers dissect and reproduce the bug in straight JS, or be a more accessible way for affected users to decrypt their key.

If we don't write bug emulation in JavaScript, it would be nice to at least document how to build a system that can reproduce the bug, ideally in a way that will work 5+ years from now.

[cantonbecker]

Hi there,

We figured out a way to make Safari 6.05 behave as it ought to, but we don’t have a way to make newer versions of Safari produce the wonky behavior. Your best option is just to install an old version the Mac OS on a computer — or even just as a virtual PC using VM Ware. Your coins are definitely not lost — it’s just a matter of finding an older mac OS to run the decryption that once.

• Canton

On Jul 26, 2015, at 2:15 PM, abkaya [email protected] wrote:

This may be too late to reopen the subject, but I have this particular issue. I saved a couple bitcoins over 4 wallets generated and BIP-38 encrypted in safari 6 at the time.

Apple doesn't offer a way to revert to safari 6.05. Would you have any suggestions for me? It would be saddening to see it lost, because of this.

Thank you for the work you guys put into it nonetheless. At least now, I've got an idea as to why the passphrase was 'wrong'.

— Reply to this email directly or view it on GitHub.

---

Canton Becker [email protected] • (505) 570-0635 • http://cantonbecker.com

[abkaya]

Thank you all for your responses and help. I currently haven't had success with BrowserStack, but I will try to reproduce the original testcase.

The error I get is: "The text you entered is not a private key", that is after I enter the password to decrypt BIP38

[scintill]

I've been working on this today, and I believe it's reproducing on a fresh install of OS X 10.7.5 (11G63) on a "MacBook4,1" model with this WebKit nightly build (the SHA256 hash of what I have is bd4751d6287e280cbf7cce6d4e9c3cc2d542b251b565b961ba574142bb54db82). @abkaya, if you can, you might try setting that up and attempting decrypting with it.

I'm not certain it's the exact same issue, because the testcases given here don't seem to work/not-work as described, on the current bitaddress.org. But I am seeing differences in my own minimal testcases (turned the scrypt parameters down to something that computes fast but often incorrect in Safari.) It does not seem entirely deterministic, either -- I am seeing 2-4 possible wrong results, plus the correct result, so maybe that has something to do with my not being able to use others' testcases. (I never open the JS console, as we've discovered that changes the behavior of the JS VM.) I'm afraid it could also be that the bitaddress.org code has changed enough to make the buggy JS engine compute differently than it did back then. So, @abkaya, be thinking of exactly which version/date you used. If you have the public addresses, looking at the blockchain timestamps may help.

Thanks to artiomchi's work, I'm comparing the execution of salsa20_8() between Chrome and the above WebKit nightly. I'm seeing patterns of certain computations being slightly off (values in B32; they differ by less than 256, which is probably a clue since 4 bytes are being assembled there), and it seems to cascade from there due to the bit rotations etc. I'm still trying to determine what exactly the pattern is, and hoping I can emulate it for the full BIP38 scrypt in other browsers. Here is an example of what I'm looking at, although you'd have to see the test code to really know what's going on there. I can share it if anyone's interested. I believe the first difference is due to the JIT bug, and the following are the cascading effects. It seems to usually be B32[0] that is wrong, although not every computation of B32[0] is wrong.

There's also a corresponding Webkit nightly source tarball that I may try building to dig deeper into what exactly the VM is doing.

[marijnfs]

Any updates on this issue? I'm trying to see this might be the problem I'm having as well.

[artiomchi]

There's nothing really that we could do at this point. It's a bug in an older version of Safari, and there's no way to port the broken code to newer browsers (for the sake of decrypting old wallets encrypted with it), since Safari is closed source.

If you think you have an old wallet encrypted with that version of Safari, you know the password and the current page can't decrypt it, your best bet is to get a VM with that version of Safari, decrypt your wallet and move your funds elsewhere asap

[artiomchi]

I said "no way to port the broken code" because it's no easy task. It's possible in theory, but as far as I know there has been no success so far

[marijnfs]

Ok i'm trying to get osx to run in virtualbox, sadly it seems quite hard to do. Do you know of any other way to run the old safari?