Z-Push icon indicating copy to clipboard operation
Z-Push copied to clipboard
verified publisher icon Z-Hub

iPhone find not returning emails

Open delacroix1983 opened this issue 8 months ago • 11 comments

Describe the issue When using the search function on iPhone (iOS 18.4) the find function gets called. It searches for the term, but does not return the value. Server behind is Dovecot / Postfix combination.

To Reproduce Open iPhone, start searching.

Expected behavior Give back the top 100 results

Screenshots No screenshot

Smartphone (please complete the following information):

  • Device: iPhone 14 Pro
  • OS: iOS 18.4
  • Apple Mail

Additional context 04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] DeviceManager->GetBackendIdForFolderId(): no backend-folderid available for '028864bd', returning as is. 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults() 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults(): Do FIND 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFindRestriction(): extracted words: Mini 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getImapIdFromFolderId('028864bd') = INBOX 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: Filter < TEXT "Mini"> 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: Recursive search INBOX 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('Drafts') = 73224fe1 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('Trash') = 08733c2c 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: SubSearch in Trash : 7 ocurrences 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('Archive') = 4f5e1d58 07/04/2025 08:04:14 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('Sent Messages') = 9d94e95c 07/04/2025 08:04:15 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: SubSearch in Sent Messages : 486 ocurrences 07/04/2025 08:04:15 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('Junk') = baf76332 07/04/2025 08:04:15 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: SubSearch in Junk : 3 ocurrences 07/04/2025 08:04:15 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('Notes') = 64f58f38 07/04/2025 08:04:15 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getFolderIdFromImapId('INBOX') = 028864bd 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: SubSearch in INBOX : 2256 ocurrences ...... 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: 1 90 9d94e95c 3571 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: 1 91 9d94e95c 3573 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMailboxSearchResults: 1 92 9d94e95c 3574 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O Find:Response 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O ItemOperations:Store 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O Mailbox 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O </ItemOperations:Store> 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O Find:Status 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O 1 07/04/2025 08:04:17 [193215] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] O </Find:Status> 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ContentParameters->GetBodyPreference(): bodypref is empty or not set 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->GetMessage('', '08733c2c:2777', '') 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getImapIdFromFolderId('') = not found 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->StatMessage('','08733c2c:2777') 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->getImapIdFromFolderId('') = not found 07/04/2025 08:04:17 [193215] [WARN] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->StatMessage('','08733c2c:2777'): Failed to retrieve overview: UID sequence syntax error 07/04/2025 08:04:17 [193215] [ INFO] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] StatusException: BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend - code: 8 - file: /usr/local/lib/z-push/lib/default/diffbackend/diffbackend.php:134 07/04/2025 08:04:17 [193215] [FATAL] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] WBXML 10K debug data: AwFqAAAZRUYDNDMyNkI3NTQtQUZENC00ODYyLUI1QzktNUM1MEY2ODVFNjlGAAFHSEkAAFADRW1haWwAAVIDMDI4ODY0YmQAAQAZSwN0bzoiTWluaSIgT1IgY2M6Ik1pbmkiIE9SIGZyb206Ik1pbmkiIE9SIHN1YmplY3Q6Ik1pbmkiIE9SICJNaW5pIiBPUiAiTWluaSIAAQFMTQMwLTk5AAEOAQEBAQ== 07/04/2025 08:04:17 [193215] [ INFO] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] User-agent: 'Apple-iPhone15C2/2205.240' 07/04/2025 08:04:17 [193215] [FATAL] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] Exception: (StatusException) - BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ZPush::PrintZPushLegal() 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] LoopDetection->ProcessLoopDetectionTerminate() 07/04/2025 08:04:17 [193215] [ INFO] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] cmd='Find' memory='1.86 MiB/2.00 MiB' time='2.25s' devType='iPhone' devId='9hun9hrnnh4m7778vs1lvgu1c0' getUser='[email protected]' from='xx.xx.xx.xx' idle='0s' version='2.7.3 ' method='POST' httpcode='500' 07/04/2025 08:04:17 [193215] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] -------- End 07/04/2025 08:04:17 [193215] [WARN] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] Unknown:0 PHP Request Shutdown: SECURITY PROBLEM: insecure server advertised AUTH=PLAIN (errflg=1) (8) 07/04/2025 08:04:17 [193215] [WARN] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] Unknown:0 PHP Request Shutdown: UID sequence syntax error (errflg=2) (8) 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] -------- Start 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] cmd='FolderSync' devType='iPhone' devId='9hun9hrnnh4m7778vs1lvgu1c0' getUser='[email protected]' from='xx.xx.xx.xx' version='2.7.3 ' method='POST' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] Used timezone 'Europe/Budapest' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] Request::ProcessHeaders() ASVersion: 16.1 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ZPush::CommandNeedsProvisioning(9): true 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ASDevice data loaded for user: '[email protected]' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] InterProcessData:__construct type: '9hun9hrnnh4m7778vs1lvgu1c0' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] TopCollector(): Initialized. 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] TopCollector initialised with IPC provider 'IpcSharedMemoryProvider' with type '20' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] InterProcessData:__construct type: '9hun9hrnnh4m7778vs1lvgu1c0' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] LoopDetection(): Initialized. 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] LoopDetection initialised with IPC provider 'IpcSharedMemoryProvider' with type '1337' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ZPush::HierarchyCommand(9): true 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] DeviceManager->ProvisioningRequired('880756496') saved device key '880756496': false 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] DeviceManager->getPolicyName(): determined policy name: 'default' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] DeviceManager->getProvisioningPolicies(): loaded 'default' policy. 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ZPush::CommandNeedsAuthentication(9): true 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] BackendIMAP->Logon(): User '[email protected]' is authenticated on '{localhost:143/imap/notls/norsh}' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ZPush::CommandNeedsPlainInput(9): false 07/04/2025 08:04:17 [228488] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] I FolderHierarchy:FolderSync 07/04/2025 08:04:17 [228488] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] I FolderHierarchy:SyncKey 07/04/2025 08:04:17 [228488] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] I {5c7d8c86-60f7-4ab0-8f06-e05ecd07f28c}1 07/04/2025 08:04:17 [228488] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] I </FolderHierarchy:SyncKey> 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] StateManager->loadHierarchyCache(): '9hun9hrnnh4m7778vs1lvgu1c0-5c7d8c86-60f7-4ab0-8f06-e05ecd07f28c-hc-1' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] FileStateMachine->getStateFiles() reading state files of '/var/lib/z-push/0/c/9hun9hrnnh4m7778vs1lvgu1c0-5c7d8c86-60f7-4ab0-8f06-e05ecd07f28c-hc*' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] FileStateMachine->getStateFiles() reading state files of '/var/lib/z-push/0/c/9hun9hrnnh4m7778vs1lvgu1c0-5c7d8c86-60f7-4ab0-8f06-e05ecd07f28c*' 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] SyncParameters->UseCPO('DEFAULT') 07/04/2025 08:04:17 [228488] [DEBUG] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] ASDevice->GetHierarchyCache(): HierarchyCache is up - Cached objects: 7 07/04/2025 08:04:17 [228488] [WBXML] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] I </FolderHierarchy:FolderSync> 07/04/2025 08:04:17 [228488] [WBXMLSTACK] [[email protected]] [9hun9hrnnh4m7778vs1lvgu1c0] WBXMLDecoder->getElementStartTag(): unmatched WBXML tag: 'FolderHierarchy:Changes' matching '' type '2' flags ''

Thank you

delacroix1983 avatar Apr 07 '25 06:04 delacroix1983

Which version of Z-Push are you running?

iredmail avatar Apr 07 '25 07:04 iredmail

Latest, downloaded today the development version.

delacroix1983 avatar Apr 07 '25 07:04 delacroix1983

Do you have FTS (Full Text Search) set up for dovecot?

matidau avatar Apr 07 '25 08:04 matidau

Yes, Xapian FTS

delacroix1983 avatar Apr 07 '25 08:04 delacroix1983

Any idea?

delacroix1983 avatar Apr 08 '25 19:04 delacroix1983

I am having the same issue while testing the latest development version. I'm using fts-flatcurve 1.0.5 with dovecot 2.3.21.1

I can add another log if it would be helpful, but I am getting the same warnings/errors:

Failed to retrieve overview: UID sequence syntax error
StatusException: BackendDiff->Fetch('%s','%s'): Error, unable retrieve message from backend - code: 8 - file: /usr/local/www/z-push/lib/default/diffbackend/diffbackend.php:134

sn3ak avatar Apr 14 '25 18:04 sn3ak

I had the same problem. If you can you could try this patch:

https://github.com/Intermesh/Z-Push/commit/229c56d4cec3f6678742f61f5205bd1dcf36d87b

mschering avatar May 15 '25 12:05 mschering

The patch improves things. It's slow to load (compared to webmail on same server. webmail is near instant, z-push took 30-45 seconds I would guess), so initially I didn't think it worked, but it does.

I get the list of found emails, but anything outside the sync period appears not to load. I let it sit there for several minutes with each message I tried.

Definitely a step in the right direction! thank you for the patch

sn3ak avatar May 17 '25 20:05 sn3ak

OK, for me it does load mails outside the sync period. Do you get any errors / info in the z-push log?

mschering avatar May 19 '25 06:05 mschering

Try my latest pull request. Then it should work.

mschering avatar May 20 '25 10:05 mschering

I haven't had a chance to look at logs, I did try your latest patch though.

~95% of the returned results loads the email. The remaining emails are listed as the message hasn't been downloaded. I waited a few minutes after the other mail populated and then gave up.

Great work! thank you for your efforts.

I will hopefully get a chance to look at logs tonight or tomorrow night.

sn3ak avatar May 20 '25 19:05 sn3ak