Probe **BROKEN** errors

[grubles]

trafficstars

Running the probe plugin hangs and results in this output in the logs.

2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py: Traceback (most recent call last):
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/pool/base.py\", line 680, in _finalize_fairy                                                                                                                                         
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:     fairy._reset(pool)                                                                           2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/pool/base.py\", line 867, in _r
eset                                                                                                                                                  
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:     pool._dialect.do_rollback(self)                                     
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py\", line 502, 
in do_rollback                                                                                                                                        2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:     dbapi_connection.rollback()                  
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py: sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140735340867824 and this is thread id 140735453678336.
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py: sqlalchemy.pool.impl.NullPool: Exception closing connection <sqlite3.Connection object at 0x7fff85022c60>                                                                                                                                              
2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py: Traceback (most recent call last):                                                               2021-02-24T16:12:05.908Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/pool/base.py\", line 680, in _f
inalize_fairy                                                                                                                                         
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:     fairy._reset(pool)            
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/pool/base.py\", line 867, in _r
eset
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:     pool._dialect.do_rollback(self)
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py\", line 502, 
in do_rollback
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:     dbapi_connection.rollback()
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py: sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread
. The object was created in thread id 140735340867824 and this is thread id 140735453678336.
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py: 
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py: During handling of the above exception, another exception occurred:
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py: 
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py: Traceback (most recent call last):
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/pool/base.py\", line 270, in _c
lose_connection
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:     self._dialect.do_close(connection)2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:   File \"/home/user/.local/lib/python3.9/site-packages/sqlalchemy/engine/default.py\", line 508, 
in do_close
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py:     dbapi_connection.close()
2021-02-24T16:12:05.909Z **BROKEN** plugin-probe.py: sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread
. The object was created in thread id 140735340867824 and this is thread id 140735453678336.
2021-02-24T16:12:06.109Z DEBUG   gossipd: Trying to find a route from (me) to 0379e31eb521ca395e4e0b42d4fff1985408f5e670085c3beaafa7260f1efcbe84 for 1
0000msat
2021-02-24T16:12:06.109Z DEBUG   gossipd: REPLY WIRE_GOSSIPD_GETROUTE_REPLY with 0 fds
2021-02-24T16:12:06.109Z INFO    plugin-probe.py: 2021-02-24 11:12:06,109 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
2021-02-24T16:12:06.109Z INFO    plugin-probe.py: sqlalchemy.engine.base.Engine: BEGIN (implicit)
2021-02-24T16:12:06.110Z INFO    plugin-probe.py: 2021-02-24 11:12:06,109 INFO sqlalchemy.engine.base.Engine INSERT INTO probes (destination, route, e
rror, erring_channel, failcode, payment_hash, started_at, finished_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
2021-02-24T16:12:06.110Z INFO    plugin-probe.py: sqlalchemy.engine.base.Engine: INSERT INTO probes (destination, route, error, erring_channel, failco
de, payment_hash, started_at, finished_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
2021-02-24T16:12:06.110Z INFO    plugin-probe.py: 2021-02-24 11:12:06,110 INFO sqlalchemy.engine.base.Engine ('0379e31eb521ca395e4e0b42d4fff1985408f5e
670085c3beaafa7260f1efcbe84', None, None, None, -1, None, '2021-02-24 11:12:06.080153', None)
2021-02-24T16:12:06.110Z INFO    plugin-probe.py: sqlalchemy.engine.base.Engine: ('0379e31eb521ca395e4e0b42d4fff1985408f5e670085c3beaafa7260f1efcbe84'
, None, None, None, -1, None, '2021-02-24 11:12:06.080153', None)
2021-02-24T16:12:06.110Z INFO    plugin-probe.py: 2021-02-24 11:12:06,110 INFO sqlalchemy.engine.base.Engine COMMIT
2021-02-24T16:12:06.110Z INFO    plugin-probe.py: sqlalchemy.engine.base.Engine: COMMIT

[cdecker]

Yep, multithreading doesn't really mesh too nicely with sqlalchemy. Been trying to squash this for a while, but it's not easy to get rid of the issue.