[Python Bindings] Dead EvenThread on exception in hook implementations

[adbenitez]

if the hook implementations of 3rd parties have bugs/exceptions, the EventThread thread dies and stops the processing of events leading to a broken state where the account doesn't shutdown but EventThread is dead.

An example bot plugin that echoes back the text sent, but that kills the event thread with a "bug" when it receives a message with text "die":

# content of echo_and_die.py based in echo_and_quit.py from https://github.com/deltachat/deltachat-core-rust/blob/master/python/examples/echo_and_quit.py

from deltachat import account_hookimpl, run_cmdline


class EchoPlugin:
    @account_hookimpl
    def ac_incoming_message(self, message):
        print("process_incoming message", message)
        if message.text.strip() == "die":
            raise ValueError('404 Life not Found')
        else:
            # unconditionally accept the chat
            message.create_chat()
            message.chat.send_text(message.text)


if __name__ == "__main__":
    run_cmdline(argv=argv, account_plugins=[EchoPlugin()])

to reproduce the problem run that script, then:

• send a "hello" message to the bot, you will get "hello" back.
• send "die" to the bot, the bot is now in "frozen" state
• send "hello" again, the bot will not reply anymore.

The issue is in: deltachat.events.EventThread._inner_run() in line 224

[adbenitez]

besides the errors that can be caused by my plugin's hook implementations and that kill the EventThread, I am getting this internal error with the latest core

[link2xt]

Duplicate with some more details: #2800

[r10s]

Duplicate with some more details: https://github.com/deltachat/deltachat-core-rust/issues/2800

as the duplicate is fixed, i assume also this issue can be closed.

[link2xt]

This is not fixed. #2800 contains two problems, one of which is fixed (something with chat ID 0) and the other one (duplicate of this one, problem with third-party code throwing expection, e.g. bots) is not.

[adbenitez]

Python bindings are crashing, today one of my bots stopped processing events, probably this same old unsolved issue of insecure _inner_run() not capturing exceptions and recovering from errors to keep event processing)

• Logs:

Exception in thread events:                                                                                         
Traceback (most recent call last):                                                                                  
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner                                             
    self.run()                                                                                                      
  File "/home/tgbridge_testrun.org/.venv/lib/python3.9/site-packages/deltachat/events.py", line 240, in run             self._inner_run()                                                                                               
  File "/home/tgbridge_testrun.org/.venv/lib/python3.9/site-packages/deltachat/events.py", line 265, in _inner_run  
    for name, kwargs in self._map_ffi_event(ffi_event):                                                               File "/home/tgbridge_testrun.org/.venv/lib/python3.9/site-packages/deltachat/events.py", line 294, in _map_ffi_eve
nt                                                                                                                  
    msg = account.get_message_by_id(ffi_event.data2)                                                                
  File "/home/tgbridge_testrun.org/.venv/lib/python3.9/site-packages/deltachat/account.py", line 423, in get_message
_by_id                                                                                                              
    return Message.from_db(self, msg_id)                                                                              File "/home/tgbridge_testrun.org/.venv/lib/python3.9/site-packages/deltachat/message.py", line 53, in from_db     
    return cls(account, ffi.gc(lib.dc_get_msg(account._dc_context, id), lib.dc_msg_unref))                          
  File "/home/tgbridge_testrun.org/.venv/lib/python3.9/site-packages/deltachat/message.py", line 26, in __init__        assert dc_msg != ffi.NULL                                                                                       
AssertionError

[link2xt]

Not fixing this issue (adding a workaround for exception), but made a PR to return None instead of raising an exception if the message does not exist: #4020

[adbenitez]

notice that the bug is completely internal to the deltachat bindings not 3rd party code, returning None doesn't fix the internal issue, just change the type of exception from AssertionError to AttributeError: 'NoneType' object has no attribute X, the None value needs to be handled otherwise #4020 wouldn't solve even the issue of getting exception when dc_get_msg returns NULL