Message duplication

[chrisballinger]

See intermittently when talking with a contact on ejabberd (with our server on prosody), on both incoming and outgoing plaintext messages.

[weiss]

I see message duplication as well. This is unrelated to the sender's server. It happens when the recipient was offline while receiving messages. ChatSecure goes online and then receives the messages both from offline storage and from MAM, and fails to deduplicate them. (This bug is hidden by enabling OMEMO or OTR, as the messages can only be decrypted once.)

The deduplication logic seems to classify a message as a duplicate only if message.threadId == buddyUniqueId is true. Maybe this is the problem? I don't understand the intention behind this check.

[ge0rg]

The issue description is utterly vague, so not sure if it is this issue or not. But I'm experiencing a similar problem to @weiss, with prosody as the server and ChatSecure as the only XMPP client used by the user. No encryption involved, most incoming messages are duplicated.

edit: This really sounds like ChatSecure is both getting messages from offline storage and from MAM.

[tmolitor-stud-tu]

I get duplicates, too, every time ChatSecure reconnects to the xmpp server.

[Letterus]

Yep, I can confirm this issue. Offline messages (MUC) get duplicated each time I connect. Makes ChatSecure hard to use.

[TixieSalander]

Same here

[ge0rg]

Tested with build 129 (4.3.1):

• sometimes messages still get duplicated when ChatSecure is offline (but not always?)
• now I get five "New Message" notifications per a single message, when the app was closed, or those five and in addition a sixth one with the actual body if the app is open.

Edit: there are five pubsub.chatsecure.org registrations with different UUIDs on that account, which probably explains why there are five notifications. I've created this account for the sole purpose of testing ChatSecure, and I only ever logged into it from this one iDevice with this one ChatSecure installation.

[weiss]

Deduplication of offline vs. MAM messages still doesn't work for me with build 134 (4.3.4) either.

I still don't understand why messages are only classified as dups if foundMessage.threadId == buddyUniqueId is true. Why isn't this a trivial message.stanzaId comparison for incoming and message.originId comparison for outgoing messages? The spec guarantees uniqueness of stanza IDs, origin IDs are controlled by yourself, why do you need additional checks?

[chrisballinger]

I think originally it was written without oid/sid support, and it would discard duplicate elementIds sent by the same buddy, which turns out was a violation of the spec anyway. There is still a bug with how oid/sid are being stored in the database, which is why you'll see the query is searching elementId matching oid also.

Without the check that the threadId matches, you could still have elementId collisions across message threads. It should probably at least be changed to let threadId = message.threadId though.

[ge0rg]

Issue still exists in 4.3.5 (135) from TestFlight.

[sgarringer]

Wondering if any progress is being made on this bug? Prosody 0.10 and latest ChatSecure on iOS and every time I reconnect messages are duplicated.

[robled]

I've experienced this problem consistently since I switched from Android (Conversations client) to iOS (ChatSecure client). The servers I connect to run ejabberd, and I found that when MUCs turn on the "archive" setting, then the dupes stop happening. I hope this info helps since it seems to be related to settings specific to each MUC on a server.