Editing of messages shared via bot inline queries, breaks for some users

[Nukesor]

Hey @Lonami I reported the issue yesterday in your chat, and created a minimal example and steps on how to reproduce the issue.

Checklist

• [x] The error is in the library's code, and not in my own.
• [x] I have searched for this issue before posting it and there isn't a duplicate.
• [x] I ran pip install -U https://github.com/LonamiWebs/Telethon/archive/master.zip and triggered the bug in the latest version.

Code that causes the issue I wrote a minimal example that can be found over here: https://github.com/nukesor/debug

1. Clone
2. poetry install
3. poetry run python main.py
4. Insert your bot credentials api_id/api_hash into debug.toml
5. poetry run python main.py
6. Send a inline query. Be aware, it doesn't fail for every user.
7. Feel free to request a message from @dasMerlon (that's also her telegram handle). She agreed on being contacted by you and it fails consistently for her.

Traceback

Traceback (most recent call last):
  File "/home/nuke/repos/ultimate_pollbot/pollbot/helper/update.py", line 139, in update_reference
    await client.edit_message(
  File "/home/nuke/.cache/pypoetry/virtualenvs/ultimate-pollbot-Jkq_D4LD-py3.8/lib/python3.8/site-packages/telethon/client/messages.py", line 1028, in edit_message
    return await self(functions.messages.EditInlineBotMessageRequest(
  File "/home/nuke/.cache/pypoetry/virtualenvs/ultimate-pollbot-Jkq_D4LD-py3.8/lib/python3.8/site-packages/telethon/client/users.py", line 74, in __call__
    result = await future
telethon.errors.rpcerrorlist.MessageIdInvalidError: The specified message ID is invalid or you can't do that operation on such message (caused by EditInlineBotMessageRequest)
The specified message ID is invalid or you can't do that operation on such message (caused by EditInlineBotMessageRequest)

The original message from your support chat:

Over the course of the last two weeks, I migrated @ultimate_pollbot from PTB to Telethon and I'm happy to announce that the rewrite went live today.

However, I'm getting some very weird Telegram exceptions in production. The bot is running as a Telegram Bot. Telethon version is 1.11.3, Python 3.8

For some users (there's no apparent pattern), editing of messages shared via inline query fail with this error:

The specified message ID is invalid or you can't do that operation on such message (caused by EditInlineBotMessageRequest)

After some trial and error, I also found that directly editing via Inline Button CallbackEvents from such messages via event.edit('some text') fail with the exact same error.

I investigated events from those messages in all affected event handlers, and id/dc_id/access_hash are the same in every handler/event.

The error occurs in chats, private chats and supergroups. But there is absolutely no pattern. For some people it doesn't work at all (deterministically), for some it only works in certain chats, for some it works all the time (for instance me).

The inline query message is an article.

[Lonami]

For what it's worth, this consistently works:

from telethon import utils

@bot.on(events.CallbackQuery)
async def handle_callback_query(event: events.CallbackQuery.Event):
    msg_id = event.original_update.msg_id
    botapi_id = utils._encode_telegram_base64(struct.pack('<iqq', msg_id.dc_id, msg_id.id, msg_id.access_hash))

    print(requests.post(f'https://api.telegram.org/bot{token}/editMessageText', json={
        'inline_message_id': botapi_id,
        'text': 'Callback response text',
        "reply_markup": {'inline_keyboard': [[{
            'text': 'callback response',
            'callback_data': '100:0:0'
        }]]}
    }).json())

Of course this is not a viable alternative, but just interesting. I have no idea what bot API is doing different from us.

For the record, also the updates in use:

good = UpdateInlineBotCallbackQuery(
	query_id=46751368606555181,
	user_id=10885151,
	msg_id=InputBotInlineMessageID(
		dc_id=4,
		id=46751367562095438,
		access_hash=1458293766830883250
	),
	chat_instance=8804301798658256640,
	data=b'100:0:0',
	game_short_name=None
)
bad = UpdateInlineBotCallbackQuery(
	query_id=46751368136827932,
	user_id=10885151,
	msg_id=InputBotInlineMessageID(
		dc_id=2,
		id=1815815020588676161,
		access_hash=-178911817370034774
	),
	chat_instance=8804301798658256640,
	data=b'100:0:0',
	game_short_name=None
)

It makes no sense.

[Lonami]

Most relevant references in tdlib are:

• https://github.com/tdlib/td/blob/c407b244a086e71773e0b9f8f64d4fb0536bc447/td/telegram/cli.cpp#L2880-L2889
• https://github.com/tdlib/td/blob/c407b244a086e71773e0b9f8f64d4fb0536bc447/td/telegram/Td.cpp#L1136-L1159

I have also confirmed not only my library is affected.

[Nukesor]

One thing I did notice is, that all errors I've seen so far, occurred on users in datacenter 2. Users in datacenter 4 however consistently work. I'm not sure whether this is an coincidence, but it's at least consistent in my test set (and as far as I can see from your example, in yours as well)

It doesn't make much sense either, though.

[Nukesor]

@Lonami Do you want to create an upstream issue on TDlib? I got a feeling you're better qualified for writing this ticket. Otherwise I'll try to come up with a proper description myself.

[Lonami]

tdlib, which I believe powers bot-API, seems to handle this correctly, which is why I linked the code here for investigation. However, I couldn't find anything weird or magic in it from a quick look.

[Nukesor]

Ok. After talking to the levlam and finally understanding what he was trying to tell me, it looks like this kind of request needs to be sent to a specific datacenter.

I looked through your init connection code, but couldn't find anything regarding specification of datacenters.

TBH, I've no idea on how to do this. Their docs don't say anything about it either.

I'm confused that something like this is even needed.

[Lonami]

Oh, thanks, that's a cool finding. The library already has code to create senders to other datacenters (e.g. downloading files) so this shouldn't be too bad.

[Nukesor]

Nice. Ping me, if I can be of any help. I'll probably need a short introduction into your code base though. I'm really looking forward to seeing this fixed.

[Lonami]

This is the code where we create a sender for another DC: https://github.com/LonamiWebs/Telethon/blob/1669d800822e93b4b00be2b161500aef4c54b6c2/telethon/client/downloads.py#L44-L58

It is very important that we return it once we're done because it was only borrowed: https://github.com/LonamiWebs/Telethon/blob/1669d800822e93b4b00be2b161500aef4c54b6c2/telethon/client/downloads.py#L87-L89

So you would do something similar when clicking fails and hope for the best, after we determine the dc_id.

[Nukesor]

Hey. Could you reopen the issue?

After some testing around, a new issue occurred: As soon as users start spamming, which then results in very fast updating of messages, this issue occurs:

2020-04-04 23:59:07,400 - telethon.network.mtprotosender - INFO - Disconnecting from 149.154.167.51:443/TcpFull...
2020-04-04 23:59:07,401 - telethon.network.mtprotosender - INFO - Disconnection from 149.154.167.51:443/TcpFull complete!
2020-04-04 23:59:08,633 - telethon.network.mtprotosender - INFO - Connecting to 149.154.167.51:443/TcpFull...
2020-04-04 23:59:08,646 - telethon.network.mtprotosender - INFO - Connection to 149.154.167.51:443/TcpFull complete!
2020-04-04 23:59:08,659 - telethon.network.mtprotosender - ERROR - Unhandled error while processing msgs
Traceback (most recent call last):
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/network/mtprotosender.py", line 497, in _recv_loop
    await self._process_message(message)
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/network/mtprotosender.py", line 512, in _process_message
    await handler(message)
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/network/mtprotosender.py", line 586, in _handle_container
    await self._process_message(inner_message)
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/network/mtprotosender.py", line 512, in _process_message
    await handler(message)
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/network/mtprotosender.py", line 558, in _handle_rpc_result
    if not isinstance(reader.tgread_object(), upload.File):
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/extensions/binaryreader.py", line 120, in tgread_object
    constructor_id = self.read_int(signed=False)
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/extensions/binaryreader.py", line 37, in read_int
    return int.from_bytes(self.read(4), byteorder='little', signed=signed)
  File "/home/bot/.cache/pypoetry/virtualenvs/ultimate-pollbot-OtaieYJ4-py3.8/src/telethon/telethon/extensions/binaryreader.py", line 60, in read
    raise BufferError(
BufferError: No more data left to read (need 4, got 0: b''); last read None
2020-04-04 23:59:08,699 - telethon.client.telegrambaseclient - INFO - Disconnecting borrowed sender for DC 2
2020-04-04 23:59:59,369 - telethon.client.telegrambaseclient - INFO - Disconnecting borrowed sender for DC 2
2020-04-04 23:59:59,369 - telethon.network.mtprotosender - INFO - Disconnecting from 149.154.167.51:443/TcpFull...
2020-04-04 23:59:59,369 - telethon.network.mtprotosender - INFO - Disconnection from 149.154.167.51:443/TcpFull complete!
2020-04-05 00:00:01,364 - telethon.network.mtprotosender - INFO - Connecting to 149.154.167.51:443/TcpFull...
2020-04-05 00:00:01,383 - telethon.network.mtprotosender - INFO - Connection to 149.154.167.51:443/TcpFull complete!

A lot of those.

If the spamming is continued, the bot breaks hard. It doesn't respond at all, CTRL+C doesn't work. Only thing left to do is kill -9

[Lonami]

This is tracked by the linked issue.

[Lonami]

https://t.me/TelethonChat/341591

[cscortes]

Is there a version of Telethon that doesn't have this bug?? Or is this fixed in a production version, if so which, please?

[Lonami]

I'll close this issue assuming it works now. If it still occurs please let me know and I will reopen.