mail
mail copied to clipboard
nextcloud
Memory exhausted during thread building with large mailbox
Steps to reproduce
- Have a mailbox with 600k messages
- Synchronize the account from the CLI
Expected behavior
Sync and threading pass
Actual behavior
Sync passes, threading :boom:
[debug] account is up to date, skipping mailbox sync
[debug] Syncing 2490
[debug] Locking mailbox 2490 for new messages sync
[debug] Locking mailbox 2490 for changed messages sync
[debug] Locking mailbox 2490 for vanished messages sync
[debug] Running partial sync for 2490
[debug] partial sync 3151:INBOX - get all known UIDs took 1s. 44/317MB memory used
[debug] partial sync 3151:INBOX - get new messages via Horde took 3s. 44/317MB memory used
[debug] partial sync 3151:INBOX - persist new messages took 0s. 44/317MB memory used
[debug] partial sync 3151:INBOX - get changed messages via Horde took 0s. 44/317MB memory used
[debug] partial sync 3151:INBOX - persist changed messages took 0s. 44/317MB memory used
[debug] partial sync 3151:INBOX - get vanished messages via Horde took 0s. 45/317MB memory used
[debug] partial sync 3151:INBOX - delete vanished messages took 0s. 45/317MB memory used
[debug] partial sync 3151:INBOX took 4s
[debug] Unlocking mailbox 2490 from vanished messages sync
[debug] Unlocking mailbox 2490 from changed messages sync
[debug] Unlocking mailbox 2490 from new messages sync
[debug] Account 3151 has 619683 messages for threading
[debug] Threading 619683 messages - build ID table took 3s. 427/445MB memory used
PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20971520 bytes) in apps/mail/lib/IMAP/Threading/Container.php on line 102
Mail app version
1.13, 1.14 and main
Mailserver or service
No response
Operating system
No response
PHP engine version
No response
Web server
No response
Database
No response
Additional info
No response
After \OCA\Mail\IMAP\Threading\ThreadBuilder::buildIdTable the memory usage is already very close to the limit (427 of 512) and the process then crashes in \OCA\Mail\IMAP\Threading\ThreadBuilder::buildRootContainer. We don't do much in there, "just" assign references to the root container. But that will be one integer for every orphan message. My test account only has orphans, so that is 600k * size of a PHP int. For my 64bit system that is 600k*8B = 4.6MB.