core icon indicating copy to clipboard operation
core copied to clipboard

`msgs` table takes a lot of space for some users

Open link2xt opened this issue 1 month ago • 1 comments

HTML messages are stored by having a whole .eml blob in the unfortunately named "mime_headers" column of the msgs table.

There are user reports that msgs table takes a lot of space since introduction of https://github.com/chatmail/core/pull/7486 which allowed us to see that the space is taken by msgs table. @adbenitez has some logs, can probably add the sizes of the app data, msgs table and number of rows from the log.

This is likely caused by mime_headers column. We even previously compressed it: https://github.com/chatmail/core/pull/4129

The next step for figuring it out is to find out all the cases when mime_headers is used. Can it be caused by:

  1. Just sending a message with large text?
  2. Receiving undecryptable messages?
  3. Receiving a message from the bot that uses set_html?

If we don't know what causes this, the issue can be closed by adding more information to the log, e.g. with the sum of sizes of mime_headers column and the number of rows that use it and mime_headers size breakdown so we can confirm that mime_headers is the reason.

link2xt avatar Dec 11 '25 17:12 link2xt

we had that in the past with save_mime_headers accidentally set, see https://github.com/chatmail/core/pull/5350 and https://github.com/deltachat/deltachat-ios/issues/2079#issuecomment-2002037067 - the option is removed meanwhile alltogether, but it may be old databases, of devs or "power-user" setting that before

r10s avatar Dec 11 '25 18:12 r10s