Cannot receive messages from a client (error: received omemo message that does not contain a key for this device, skipping)

[Elizafox]

One of my contacts seems to be having trouble talking to me with this showing up in the debug log:

19:50:37: (Libpurple: jabber) Recv (ssl)(1015): <message xml:lang='en' to='[email protected]/garrett' from='[email protected]/pidgin' type='chat' id='purple76c4b2d0'><active xmlns='http://jabber.org/protocol/chatstates'/><encrypted xmlns='eu.siacs.conversations.axolotl'><header sid='1606509255'><key rid='1068475319'>[REDACTED]</iv></header><payload>[REDACTED]</payload></encrypted><encryption xmlns='urn:xmpp:eme:0' namespace='eu.siacs.conversations.axolotl' name='OMEMO'/><store xmlns='urn:xmpp:hints'/></message>
19:50:37: (Libpurple: jabber) XML parser error for JabberStream 0x7ff293e4c180: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
19:50:37: (Libpurple: lurch) [AXC INFO] axc_init: initializing axolotl client
19:50:37: (Libpurple: lurch) [AXC DEBUG] axc_init: created and set axolotl context
19:50:37: (Libpurple: lurch) [AXC DEBUG] axc_init: set axolotl crypto provider
19:50:37: (Libpurple: lurch) [AXC DEBUG] axc_init: created store context
19:50:37: (Libpurple: lurch) [AXC DEBUG] axc_init: set store context
19:50:37: (Libpurple: lurch) [AXC INFO] axc_init: done initializing axc
19:50:37: (Libpurple: lurch) received omemo message that does not contain a key for this device, skipping

I have the following ID's:

This device's ([email protected]:1852516058) fingerprint: [REDACTED]
[email protected]:1879055436's fingerprint: [REDACTED]
[email protected]:1606509255's fingerprint: [REDACTED]
[email protected]:2015452324's fingerprint: [REDACTED]

I see nothing else in the debug log indicating anything is amiss.

[gkdr]

Which client is your contact using? The key snippet is <key rid='1068475319'>, rid being the recipient ID. Since it is the only <key> element, this means that the message is only encrypted for one device by your contact, and this device's ID matches neither of your or your contact's devices.

[Elizafox]

They're also using pidgin

Note, that's what shows up in my debug log

[gkdr]

On your side, everything is correct. I'm gonna need the sender's debug log.

[gkdr]

@Elizafox, does the issue still persist?

[NattyNarwhal]

I'm said contact. Gajim seems to work, but Pidgin doesn't. I get a message like:

(3:50:14 PM) Even though an encrypted session exists, the recipient's devicelist is empty.The user probably uninstalled OMEMO, so you can add this conversation to the blacklist.

(3:50:14 PM) Me: broken message 2

from Pidgin debug window:

(15:50:14) lurch: retrieved devicelist for [email protected]:
<publish node="eu.siacs.conversations.axolotl.devicelist"><item><list
xmlns="eu.siacs.conversations.axolotl" /></item></publish>

(15:50:14) lurch: retrieved own devicelist:
<publish node="eu.siacs.conversations.axolotl.devicelist"><item><list
xmlns="eu.siacs.conversations.axolotl"><device id="470196580" /><device
id="541634781" /><device id="2111591022" /></list></item></publish>

(15:50:14) lurch: lurch_msg_encrypt_for_addrs: trying to encrypt key for 2 devices
(15:50:14) lurch: lurch_key_encrypt: encrypting key for [email protected]:2111591022
(15:50:14) lurch: lurch_key_encrypt: encrypting key for [email protected]:470196580
(15:50:14) xmlnode: XML parser error for xmlnode 0760FCC0: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(15:50:14) jabber: Sending (ssl) ([email protected]/azimuth): <message type='chat' id='purpled04bde74' to='[email protected]'><active xmlns='http://jabber.org/protocol/chatstates'/><encrypted xmlns='eu.siacs.conversations.axolotl'><header sid='541634781'><key rid='470196580'>MwohBU2pc/zLChknR1gpFSAm5CQ6tq8uBGVfLk3uZ4ErD4cVEAUYBSIwU482vFtKOCbwf+DiZcJfzu9pWB/PfY6A06TJQ31c084FbiqP7U9Qv/ZzH7dxNu3bhupK5e7OkjI=</key><key rid='2111591022'>MwohBYdzQiMq9UaBnaSani29My/moWssyljpHIKSSYsF/QZHEAEYAyIwGFXlib8AkyPxsGhf+kQyLardVXPzAJHMX9BU3pFk//B1oTAGjIJq07FIbJpO24XYInpY+3A+C9o=</key><iv>PF2uZ3JoC0olUs1ZNF7G6Q==</iv></header><payload>ih93oon8LxjROlKn2m/MiQ==</payload></encrypted><encryption xmlns='urn:xmpp:eme:0' namespace='eu.siacs.conversations.axolotl' name='OMEMO'/><store xmlns='urn:xmpp:hints'/></message>
(15:50:14) jabber: Sending (ssl) ([email protected]/azimuth): <message type='chat' id='purpled04bde75' to='[email protected]'><active xmlns='http://jabber.org/protocol/chatstates'/></message>
(15:50:51) jabber: Sending (ssl) ([email protected]/azimuth): <iq type='get' id='purpled04bde76'><ping xmlns='urn:xmpp:ping'/></iq>
(15:50:51) jabber: Recv (ssl)(69): <iq id='purpled04bde76' type='result' to='[email protected]/azimuth'/>

Interestingly, I can see messages I sent them with Conversations, but their incoming OMEMO messages only get shown by Gajim. The broken messages I send by Pidgin appear to Gajim as well.

[gkdr]

Hi @NattyNarwhal, from what I can tell in the debug log, it is correct - your contact's device list is empty, so you only see this message on your own devices (correct?). This plugin checks if there is a double ratchet session to determine whether to encrypt or not, and that seems to exist. But then the OMEMO devicelist is empty, for some reason.

PEP nodes (one of which stores the devicelist) can get randomly deleted by the server, but clients should readd themselves if they receive a devicelist they are not part of. Can you tell me what you see when you paste the following to the XMPP console? Just replace the JID.

<iq type='get'
    to='b@localhost'
    id='whatever123'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <items node='eu.siacs.conversations.axolotl.devicelist'/>
  </pubsub>
</iq>

[NattyNarwhal]

<iq type='get' to='[email protected]' id='[email protected]'>   		<pubsub xmlns='http://jabber.org/protocol/pubsub'>     				<items node='eu.siacs.conversations.axolotl.devicelist'/>

   </pubsub>

 </iq>
<iq id='[email protected]' type='result' to='[email protected]/azimuth' from='[email protected]'>
	<pubsub xmlns='http://jabber.org/protocol/pubsub'>
		<items node='eu.siacs.conversations.axolotl.devicelist'>
			<item id='5F79184261FB2'>
				<list xmlns='eu.siacs.conversations.axolotl'>
					<device id='1852516058'/>
					<device id='1619614828'/>
					<device id='1924949908'/>
				</list>
			</item>
		</items>
	</pubsub>
</iq>

[gkdr]

Thanks, that looks fine. This is while the issue persists? If so, do you have an sqlite3 client installed and are you willing to look in the DB for me? Just open the DB file in your purple home dir via sqlite3 a@localhost_omemo_db.sqlite (this is your account name) and then type select * from devicelists;. Does your contact actually not show up there?

Also, does a client restart usually resolve the issue? If not, can you leave the debug window open when you close Pidgin and restart it so you can see everything that happens when you start it up? I'm looking for something like the following: (22:09:53) lurch: lurch_devicelist_process: processing devicelist from b@localhost for b@localhost

[NattyNarwhal]

A client restart doesn't fix it.

Perhaps interesting:

(17:23:20) jabber: XML parser error for JabberStream 074E07B8: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: comparing received devicelist with cached one
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: own id was missing, adding it
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: devicelist needs publishing...
(17:23:20) xmlnode: XML parser error for xmlnode 0750B178: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: 
...done:
(17:23:20) xmlnode: XML parser error for xmlnode 075B4960: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(17:23:20) lurch: lurch_bundle_publish_own: published own bundle for [email protected]
(17:23:20) lurch: lurch_devicelist_process: processing devicelist from [email protected] for [email protected]
(17:23:20) lurch: lurch_devicelist_process: cached devicelist is
<publish node="eu.siacs.conversations.axolotl.devicelist"><item><list
xmlns="eu.siacs.conversations.axolotl"><device id="470196580" /><device
id="541634781" /><device id="2111591022" /></list></item></publish>
[...]
(17:23:20) jabber: XML parser error for JabberStream 074E07B8: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: comparing received devicelist with cached one
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: own id was missing, adding it
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: devicelist needs publishing...
(17:23:20) xmlnode: XML parser error for xmlnode 075B4F80: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(17:23:20) lurch: lurch_pep_own_devicelist_request_handler: 
...done:
(17:23:20) xmlnode: XML parser error for xmlnode 075B23E0: Domain 3, code 100, level 1: xmlns: URI eu.siacs.conversations.axolotl is not absolute
(17:23:20) lurch: lurch_bundle_publish_own: published own bundle for [email protected]
(17:23:20) lurch: lurch_devicelist_process: processing devicelist from [email protected] for [email protected]
(17:23:20) lurch: lurch_devicelist_process: cached devicelist is
<publish node="eu.siacs.conversations.axolotl.devicelist"><item><list
xmlns="eu.siacs.conversations.axolotl"><device id="470196580" /><device
id="541634781" /><device id="2111591022" /></list></item></publish>

[NattyNarwhal]

I did said query on the OMEMO DB and I didn't see the contact? I see myself and another contact, but not them.

[gkdr]

Yeah, that's consistent with what you see in the debug log. At least it's probably not a DB issue.

Somehow, the parsing of the devicelist does seem broken though. Did what you showed me just happen 3 times or does it happen over and over? Is your device ID actually in there or not? Do you happen to still have the raw XML of those messages? Do you happen to know which software your server is running?

[NattyNarwhal]

Just twice.

My server is running Prosody (latest stable, I believe); the other end is ejabberd.

[gkdr]

I'm testing against Prosody 0.10.1 locally and I never noticed such a thing, but I also didn't really pay much attention to it. As I said, if you can give me the raw XML, maybe I could look for some more pointers, but I'll have a look at the devicelist code.

When your account connects, you should receive a devicelist update for yourself + all of your OMEMO contacts, and see this in the debug window. These are handled by the lurch_devicelist_process callback I mentioned above, and if you don't see your contact there, the servers are getting the PEP handling wrong, because the node is definitely there. However, I wonder why your contact is not in the DB - you must have received an empty devicelist update, which would cause a deletion of all devices (which I should probably rethink since it seems easy to provoke this). Actually, now I'm not sure if it ever worked for you? In any case, thanks for providing all the info.

[snowkat]

In case it helps, I had a similar issue with a contact (I was the one receiving the "recipient's devicelist is empty" error). Unsubscribing and Re-requesting authorization resolved the issue, and so far is persisting across restarts.

I have no idea why OMEMO worked fine with Conversations, though. Hopefully this at least works as a workaround for now if anyone else runs into this.