Syncthing corrupts database after successful attachment of small files

[fellowish]

KeepassDX completely corrupted my database following the attachment of a 5 kb file. It was unable to allocate enough memory to load the database following this occurring, and subsequently led to the entire database being unrecoverable.

The file in question was a TXT attached to an already existing entry. The database was operating correctly beforehand.

To Reproduce:

1. Attempt to attach a file to an entry
2. Successfully attach the file to the entry
3. Close the app
4. Attempt to open the database

Expected behavior KeepassDX opening a non-corrupted database.

KeePass Database

• Created with: KeepassXC
• Version: 2.6.6
• Location: Local File sent via Syncthing
• Size: 111.45 kb
• Contains attachment: Yes. A .txt file.

KeePassDX (please complete the following information):

• Version: 3.0.2
• Build: Libre
• Language: English

Android (please complete the following information):

• Device: Pixel 5
• Version: 11

[J-Jamet]

I can't reproduce your problem, I have done many tests with the attachments and they are all OK. Can you upload a database that reproduces the problem? Have you tried to register the database with another application than KeePassXC and KeePassDX? Are you sure it doesn't work locally with a file that is not synchronized with Syncthing?

[fellowish]

I just managed to scrape the existing database from cached files in system memory, which thankfully did not have the TXT within it. I'm assuming what occurred was an update being triggered as the TXT was being parsed, causing it to be saved invalidly upon successful attachment.

I'd need to experiment with the sync trigger. I am going to note that this hasn't occurred the other way around (KeePassXC on PC), or with other applications which are compatible with .kdbx files (e.g. AuthPass).

You mentioned in other issues that you worked on decryption of large files, but this also seems to affect the encryption as well. Outside of KeepassDX I was getting header reading errors, but within KeepassDX it mentioned a memory allocation error instead. (See issue #993.) This file wasn't large. This could be pointing towards errors in file encryption/decryption or just incorrect error logging instead.

Again, I'd need to test.

[J-Jamet]

The storage operation differs for very large files, this size is calculated according to the memory available in your device, but in 98% of cases the stream of attached files stores the data in temporary memory, so a file of 5Kb or 5Mb will theoretically have the same behavior. On the other hand, if it's a 5GB file, the system will try to store it in hard memory and encrypt it on the fly (which will take a lot of time, so it's clearly not advisable to store large files) It is managed in pools, here is the package that contains the code : https://github.com/Kunzisoft/KeePassDX/tree/master/app/src/main/java/com/kunzisoft/keepass/database/element/binary

But I don't think that's where the problem lies. The exceptions thrown depend on the parsers that are implemented according to the applications, so if there are not the same errors it is just that the parser is not the same.

My first theory is that the synchronization system you set up with Syncthing breaks the data provider at some point and corrupts the data saved. If the file or header is only partially saved because the file provider interrupts the stream, the file gets corrupted.

[fellowish]

Good point.

If the error was the synchronization interrupting the data provider, why is it only occuring with KeepassDX and not other clients like AuthPass? This would make the most sense, but it's unexpected behavior.

~~KeepassDX showed the attachment as being successful, and displayed data normally until the database was locked (at which point it was inaccessible). AuthPass behaved as expected multiple times when attaching files to entries.~~

~~Is this caused by KeepassDX parsing/saving header information differently? How does this differ from AuthPass/KeepassXC? Why are they not behaving similarly?~~

EDIT: Looking at it more, it might instead stem from the database being saved while it is still open within KeepassDX. This could trigger a sync between devices which interrupts the data provider. If the database is then locked in the app, it could be attempting to save header information while the data provider becomes unavailable.

This would cause it to be corrupted in the manner you described. Then the corrupted database is synchronized, overwriting the intact database.

The difference in this case would be how AuthPass and KeepassXC save information in comparison to KeepassDX. When does the app save header data?

[J-Jamet]

The header is saved along with the other information (but first in the stream). https://github.com/Kunzisoft/KeePassDX/blob/master/app/src/main/java/com/kunzisoft/keepass/database/file/output/DatabaseOutputKDBX.kt https://github.com/Kunzisoft/KeePassDX/blob/master/app/src/main/java/com/kunzisoft/keepass/database/file/output/DatabaseHeaderOutputKDBX.kt

KeePassXC is not an android application so cannot use the Storage Access Framework with a file provider.

Authpass saves a file in the internal memory of the app, so in /data/user/0/design.codeux.authpass.fdroid/app_flutter/[database.kdbx]. The SAF content provider is only used if you select "Load from..." with "File Picker", the file location is "content://com.android.externalstorage..." I haven't studied how saving the file in the stream for a URI in Authpass works, but it shouldn't differ much from KeePassDX.

[J-Jamet]

I have tested the file backup many times and I never have a problem if the URI stream is not interrupted.

[Jrchintu]

Recently i had also got this same problem that led to corruption of my database, ths total size of database was only few kb but still the error of out of memory appears, if possible please take a look at this issue once again. Keepass version- 3.2.0

[J-Jamet]

I can look all I want, if there is no specific reproducible procedure, I don't know what I can do. As I said, if the synchronization state is wrong and Syncthing manipulates it, it can theoretically lead to this kind of situation. I've already mentioned this in the wiki.