Some items cannot be decrypted

[alexanderadam]

So like others I'm getting corrupted databases on a regular basis. Usually I'm taking the two hours and "babysit" a full sync.

But this time even doing a full sync didn't help and Joplin isn't giving any actionable advice.

I wasn't aware that Joplin isn't meant for end-users but rather for very tech-savvy people when I migrated to Joplin.

Environment

Joplin version: 2.12.2 Platform: Android OS specifics: Android 13 / One UI 5.1

Steps to reproduce

1. use the Joplin sync with encryption on mobile devices

Describe what you expected to happen

In an ideal world this would be to fix Joplin sync so that corruption doesn't happen but I gave up on that, so I would suggest that the error message would give at least actionable advice, hints or options other than a non-working Retry button.

Logfile

[laurent22]

If you have corrupted data, you have a bigger problem than the Retry button. You should probably check your server or maybe your Android device, and find out what is corrupting your data.

[alexanderadam]

What happens if you don't babysit a sync and the app is killed during a write? Is writing atomic and within a transaction so that a half-written record is safe and won't harm since the relevant records are stored separately and the transaction is aborted or will it cause issues?

The Android device works fine and literally every application except Joplin works perfectly fine. The storage is Dropbox and I'm writing backups with RoundSync and other things from this device to Dropbox too (even huge binaries). But the other services have background sync so this won't be killed if the display turns off.

Furthermore I don't know of any other application in general that produces corrupted data. 🤷‍♂️ Do you?

[laurent22]

If it's dropbox it's not corrupted data but something else. We need more info to know what it is

[semiopat]

I have the same problem. It happens after the dropbox sync began to fail. After a long search, I ended to understand that iOS versions of Joplin (I use two, iphone and ipad) create a new Joplin repertory in Dropbox>Applications, added as Joplin 1 by the system. I saved the first repertory and try several manipulations (uninstall, reinstall, etc.) I was about too loose all my data, but I finally find a way:

1. I reinstall all apps (1 macOS and 2 iOS) after erasing the Joplin folder in Dropbox>Applications
2. I let one iOS app recreate a Joplin folder after first sync.
3. I sync with macOS, which use the same folder
4. I copy all the md files of the previous folder I saved.

And it worked; I can recover all my files and a correct synchronisation behavior, but… since I have this alert on every sync, on both macOS and iOS : And this list of files that can't be download get longer and longer: If I click on Réessayer (Retry), the mention disappear, but another come back on the next sync.

Here is the debugging report, maybe it could help: syncReport-1694411239399.csv

[alexanderadam]

I have the feeling that the decryption issue that happens is not related to yours @semiopat.

@laurent22 in the log I see Error: Item is not encrypted: $hash.

I never used Joplin without encryption but maybe it's one of the default notes? But even then: if a user decides to have all notes encrypted and Joplin finds an unencrypted note, then it should be encrypted instead of raising an exception. No? :thinking:

[github-actions[bot]]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? If you require support or are requesting an enhancement or feature then please create a topic on the Joplin forum. This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[alexanderadam]

As usual, the bug is still there

[semiopat]

The bug is still there on iOS, but no longer on macOS (??)

[alexanderadam]

I never had it on my desktop either. Just on my mobile devices.

[semiopat]

What ?!! The bug came back on macOS, maybe after the sync with the iOS version that I didn’t use for a while

[github-actions[bot]]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? If you require support or are requesting an enhancement or feature then please create a topic on the Joplin forum. This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[alexanderadam]

Of course the bot is spamming around as usual while the issue is still present :laughing:

[alexanderadam]

Hello @laurent22,

I wish you well. As mentioned in the other issue I had needed the usual full sync again and of course I'm running into this issue here too.

You'll find the full sync report here:

syncReport-1703369546880.txt

[laurent22]

@alexanderadam, could you try closing the app completely (or restart your phone), then open the app again, and click "Retry all". Does it work? If not please share the log again to see why they are not getting decrypted

[alexanderadam]

Thank you for the fast response @laurent22

I did try the "Retry all" even before I opened this issue and it never worked. ~I just tried it again and error still pops up but the Sync export is now just empty?~ :thinking:

EDIT: Nevermind, the file manager just showed a wrong size:

syncReport-1703371217589.txt

[laurent22]

So these two items still appear in the status page even after restarting the phone and pressing "Retry all"?

[alexanderadam]

I didn't reboot when I attached the last log. I just killed and restarted the app.

Now I did a full reboot and hit "Retry all" and I'm still getting that message.

However, the shown failing revisions change sometimes. Now it showed the error just for fa9b9b55d1eb4a5ba727cd608ed5a8fc and hitting "retry all" keeps that info but after I pressed "Refresh" I had the two hashes that you see in the screenshot of the first comment.

PS: I'm still seeing this error exclusively on my Android phone and not on Desktop

[laurent22]

Thanks we'll look into it. I'm putting the relevant part of the log here to make it easier to find it back. I think the app is trying to decrypt an item that's already decrypted, so it should just skip it

12-23T23:39:58,20,"""DecryptionWorker: error for: 3d1898743ae34e0699957acebe9f23f4 (notes)", "Error: Item is not encrypted: 3d1898743ae34e0699957acebe9f23f4
Error: Item is not encrypted: 3d1898743ae34e0699957acebe9f23f4
    at ?anon_0_ (address at index.android.bundle:1:2576779)
    at next (native)
    at anonymous (address at index.android.bundle:1:2568049)
    at tryCallTwo (/root/react-native/ReactAndroid/hermes-engine/.cxx/MinSizeRel/5p1x2e66/arm64-v8a/lib/InternalBytecode/InternalBytecode.js:61:9)
    at doResolve (/root/react-native/ReactAndroid/hermes-engine/.cxx/MinSizeRel/5p1x2e66/arm64-v8a/lib/InternalBytecode/InternalBytecode.js:216:25)
    at Promise (/root/react-native/ReactAndroid/hermes-engine/.cxx/MinSizeRel/5p1x2e66/arm64-v8a/lib/InternalBytecode/InternalBytecode.js:82:14)
    at anonymous (address at index.android.bundle:1:2567876)
    at decrypt (address at index.android.bundle:1:2576696)
    at ?anon_0__loop (address at index.android.bundle:1:3664033)
    at next (native)
    at ?anon_0_ (address at index.android.bundle:1:3663069)
    at next (native)
    at fulfilled (address at index.android.bundle:1:3660623)
    at tryCallOne (/root/react-native/ReactAndroid/hermes-engine/.cxx/MinSizeRel/5p1x2e66/arm64-v8a/lib/InternalBytecode/InternalBytecode.js:53:16)
    at anonymous (/root/react-native/ReactAndroid/hermes-engine/.cxx/MinSizeRel/5p1x2e66/arm64-v8a/lib/InternalBytecode/InternalBytecode.js:139:27)
    at apply (native)
    at anonymous (address at index.android.bundle:1:218984)
    at _callTimer (address at index.android.bundle:1:217933)
    at _callReactNativeMicrotasksPass (address at index.android.bundle:1:218097)
    at callReactNativeMicrotasks (address at index.android.bundle:1:220072)
    at __callReactNativeMicrotasks (address at index.android.bundle:1:121378)
    at anonymous (address at index.android.bundle:1:120471)
    at __guard (address at index.android.bundle:1:121219)
    at flushedQueue (address at index.android.bundle:1:120382)
    at invokeCallbackAndReturnFlushedQueue (address at index.android.bundle:1:120325)", "{"user_data":"","master_key_id":"","conflict_original_id":"","share_id":"","is_shared":0,"encryption_applied":1,"user_updated_time":1666636039427,"application_data":"","source_application":"net.cozic.joplin-mobile","source":"joplin","is_todo":0,"markup_language":0,"order":0,"latitude":"0.00000000","encryption_cipher_text":null,"longitude":"0.00000000","is_conflict":0,"todo_completed":0,"altitude":"0.0000","parent_id":"a91351df55654736a6a8959186eb5e5b","title":"","source_url":"","updated_time":1666636039427,"id":"3d1898743ae34e0699957acebe9f23f4","todo_due":0,"author":"","user_created_time":1703327169034,"body":"","created_time":1703327169034,"type_":1}""

[alexanderadam]

Thanks we'll look into it.

Thank you

I think the app is trying to decrypt an item that's already decrypted, so it should just skip it

Yes, that's what I hinted to in the comment above

[personalizedrefrigerator]

It looks like you're using Dropbox sync.

Doing the following might be helpful. On dropbox.com, search for 3d1898743ae34e0699957acebe9f23f4 (the ID of the note that isn't syncing).

From the log, it seems to be a note, so there should be just one .md file in the search results.

In the log, the JSON data for the item has "encryption_cipher_text":null. Is this also the case on Dropbox?

(The log suggests that the item has the following JSON data:

{"user_data":"","master_key_id":"","conflict_original_id":"","share_id":"","is_shared":0,"encryption_applied":1,"user_updated_time":1666636039427,"application_data":"","source_application":"net.cozic.joplin-mobile","source":"joplin","is_todo":0,"markup_language":0,"order":0,"latitude":"0.00000000","encryption_cipher_text":null,"longitude":"0.00000000","is_conflict":0,"todo_completed":0,"altitude":"0.0000","parent_id":"a91351df55654736a6a8959186eb5e5b","title":"","source_url":"","updated_time":1666636039427,"id":"3d1898743ae34e0699957acebe9f23f4","todo_due":0,"author":"","user_created_time":1703327169034,"body":"","created_time":1703327169034,"type_":1}

[alexanderadam]

It looks like you're using Dropbox sync.

I do

Specifically, does it have "encryption_cipher_text":null?

No, encryption_cipher_text is set to a 51 character string and in the line afterwards a JSON hash comes ({"iv":"…","v":1,"iter":101," … }).

[laurent22]

Any chance you could share that complete .md file? Note that it's encrypted so we won't be able to read any of the actual content but that might help us understand why that value ends up being null in Joplin

[laurent22]

I downloaded it and deleted it

[alexanderadam]

Thank you!

[laurent22]

I looked into this but couldn't replicate. Leaving it open in case we get more information via other bug reports

[alexanderadam]

@laurent22 is there any way that I could 'repair' this manually at least with the JS console or so?

[personalizedrefrigerator]

Possibly related to https://github.com/laurent22/joplin/issues/9904

[alexanderadam]

Possibly related to #9904

Indeed, it looks like 9904 is a duplicate