Crash when trying to perform account backup (running out of memory during SQLite VACUUM)

[gerryfrancis]

• Android version: Android 11.

• Device: Fairphone FP2 (Lineage OS 18.1 build RQ3A.211001.001 1618d67eec from 2024-02-12).

• Delta Chat version: 1.43.0 (nightly built 2024-02-14).

• Expected behavior: A backup of an account can be performed.

• Actual behavior: A backup of an account cannot be performed. The app crashes and is relaunched at 1% progress. (The content of the log which I created after three failed attempts - see below - indicates that there might be a memory allocation issue.)

• Steps to reproduce the problem: The steps to create a backup from an account are obvious, and it was possible to perform the operation successfully after ~100 attempts. I have attach a log file - see below - that includes three failed tries after a reboot of the device.

• Screenshots: N/A.

• Logs: deltachat-log-20240220-093456_2.txt

[adbenitez]

hi @gerryfrancis, is this still happening to you in 1.44.0 stable release? I just tried it and could create a backup without any issues in 1.44

[gerryfrancis]

Hi @adbenitez , I can reproduce the bug with a two days old nightly build, core version 1.137.0, as well:

deltachat-log-20240405-093202_2.txt

[adbenitez]

thanks! @gerryfrancis, I am afraid this is not a bug but that you don't have enough disk space:

04-05 09:31:43.667 13558 13589 W DeltaChat: src/imex.rs:799: Vacuum failed, exporting anyway out of memory

cc @link2xt

[gerryfrancis]

I believe this could be a similar case like https://github.com/deltachat/deltachat-core-rust/pull/5349 , but for Android.

@adbenitez No, there is enough disk space left to create a backup.

[r10s]

ftr, the log says Fatal signal 6 (SIGABRT), "Abort message: 'terminating with uncaught exception of type St9bad_alloc: std::bad_alloc'", not sure if that is related to vacuum; unfortunately, i also cannot reproduce the issue

[link2xt]

bad_alloc means some C++ code failed to allocate memory. std::bad_alloc is a C++ exception: https://en.cppreference.com/w/cpp/memory/new/bad_alloc

[gerryfrancis]

bad_alloc means some C++ code failed to allocate memory.

@link2xt This error occurs with my Fairphone FP2, it features 2 GB RAM only, and the database size has been >2 GB. So, if there was a low memory issue during vacuuming, I would have expected a message like "your device has run out of memory" rather than an app crash.

On the other hand, we could implement the same fix or workaround we already did in iOS for Android, too (see my comment above), and I would test again afterwards. (I still own my Fairphone.) :)

[adbenitez]

I tried in a device with full storage and it doesn't crash:

bad_alloc means some C++ code failed to allocate memory.

so it is about RAM and an issue in the low level core side, so nothing we can solve here in the android side, moving to core

[link2xt]

SQLite is written in C rather than C++, so it is not even an SQLite crash, but some C++ component in Android application failing. But memory is exhausted by SQLite I guess.

[link2xt]

Closing as it is apparently not even reproducible anymore and looks like a generic out of memory crash.

There have been changes to how we run checkpointing, maybe the issue was a large WAL log and is solved by this: #7096

If it is reported again for some recent version, we might look into not running VACUUM, but I don't think it is feasible to not run VACUUM at all as the database will only grow.