bbot icon indicating copy to clipboard operation
bbot copied to clipboard

Gowitness sqlite query blocking event loop

Open TheTechromancer opened this issue 1 year ago • 1 comments

It looks like the gowitness module is blocking the event loop with its sqlite queries. Normally we would want to asyncify sqlite, but in this case we will also want to make sure the queries are optimized so they don't take so long to complete.

@Sh4d0wHunt3rX

TheTechromancer avatar Oct 22 '24 17:10 TheTechromancer

GDB traceback:

(gdb) where
#0  0x000055db893ab68f in ?? ()
#1  0x000055db893ac91e in PyImport_AddModuleObject ()
#2  0x000055db893ac8e1 in PyImport_AddModule ()
#3  0x000055db893c919a in PyRun_SimpleStringFlags ()
#4  <function called from gdb>
#5  futex_wait (private=0, expected=2, futex_word=0x55db89716c70 <_PyRuntime+432>) at ../sysdeps/nptl/futex-internal.h:146
#6  __GI___lll_lock_wait (futex=futex@entry=0x55db89716c70 <_PyRuntime+432>, private=0) at ./nptl/lowlevellock.c:49
#7  0x00007ff18d940002 in lll_mutex_lock_optimized (mutex=0x55db89716c70 <_PyRuntime+432>) at ./nptl/pthread_mutex_lock.c:48
#8  ___pthread_mutex_lock (mutex=0x55db89716c70 <_PyRuntime+432>) at ./nptl/pthread_mutex_lock.c:93
#9  0x000055db8928eeaf in ?? ()
#10 0x000055db892c4046 in PyEval_RestoreThread ()
#11 0x000055db893c3df0 in PyGILState_Ensure ()
#12 <function called from gdb>
#13 futex_wait (private=0, expected=732706, futex_word=0x55db89716c60 <_PyRuntime+416>) at ../sysdeps/nptl/futex-internal.h:146
#14 futex_wait_simple (private=0, expected=732706, futex_word=0x55db89716c60 <_PyRuntime+416>) at ../sysdeps/nptl/futex-internal.h:177
#15 __condvar_acquire_lock (private=0, cond=0x55db89716c40 <_PyRuntime+384>, cond@entry=0x0) at ./nptl/pthread_cond_common.c:140
#16 __condvar_cancel_waiting (cond=cond@entry=0x55db89716c40 <_PyRuntime+384>, seq=seq@entry=2161590, g=g@entry=1, private=private@entry=0) at ./nptl/pthread_cond_wait.c:85
#17 0x00007ff18d93c02f in __pthread_cond_wait_common (abstime=0x7fff6774f390, clockid=1, mutex=0x55db89716c70 <_PyRuntime+432>, cond=0x55db89716c40 <_PyRuntime+384>)
    at ./nptl/pthread_cond_wait.c:516
#18 ___pthread_cond_timedwait64 (cond=0x55db89716c40 <_PyRuntime+384>, mutex=0x55db89716c70 <_PyRuntime+432>, abstime=0x7fff6774f390) at ./nptl/pthread_cond_wait.c:652
#19 0x000055db8928f040 in ?? ()
--Type <RET> for more, q to quit, c to continue without paging--
#20 0x000055db892c4046 in PyEval_RestoreThread ()
#21 0x000055db893c3df0 in PyGILState_Ensure ()
#22 <function called from gdb>
#23 __futex_abstimed_wait_common64 (private=1735717919, cancel=true, abstime=0x7fff6774f5f0, op=137, expected=0, futex_word=0x55db89716c68 <_PyRuntime+424>) at ./nptl/futex-internal.c:57
#24 __futex_abstimed_wait_common (cancel=true, private=1735717919, abstime=0x7fff6774f5f0, clockid=1, expected=0, futex_word=0x55db89716c68 <_PyRuntime+424>) at ./nptl/futex-internal.c:87
#25 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55db89716c68 <_PyRuntime+424>, expected=expected@entry=0, clockid=clockid@entry=1, 
    abstime=abstime@entry=0x7fff6774f5f0, private=private@entry=0) at ./nptl/futex-internal.c:139
#26 0x00007ff18d93be9b in __pthread_cond_wait_common (abstime=0x7fff6774f5f0, clockid=1, mutex=0x55db89716c70 <_PyRuntime+432>, cond=0x55db89716c40 <_PyRuntime+384>)
    at ./nptl/pthread_cond_wait.c:503
#27 ___pthread_cond_timedwait64 (cond=0x55db89716c40 <_PyRuntime+384>, mutex=0x55db89716c70 <_PyRuntime+432>, abstime=0x7fff6774f5f0) at ./nptl/pthread_cond_wait.c:652
#28 0x000055db8928f040 in ?? ()
#29 0x000055db892c4046 in PyEval_RestoreThread ()
#30 0x00007ff1800551f8 in ?? () from /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so
#31 0x000055db892b9caf in _PyEval_EvalFrameDefault ()
#32 0x000055db892dca5e in ?? ()
#33 0x000055db892cf221 in _PyObject_GenericGetAttrWithDict ()
#34 0x000055db892cd6fd in PyObject_GetAttr ()
#35 0x000055db892befc1 in _PyEval_EvalFrameDefault ()
#36 0x000055db892ee8b0 in ?? ()
#37 0x000055db892bb904 in _PyEval_EvalFrameDefault ()
#38 0x000055db892ee8b0 in ?? ()
#39 0x000055db892bb904 in _PyEval_EvalFrameDefault ()
#40 0x000055db892ee8b0 in ?? ()
#41 0x00007ff18c5c928e in ?? () from /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so
#42 0x00007ff18c5ca49b in ?? () from /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so
#43 0x000055db892d0574 in ?? ()
#44 0x000055db893ad505 in ?? ()
#45 0x000055db89429e82 in ?? ()
#46 0x000055db892c420b in ?? ()
#47 0x000055db892bba6e in _PyEval_EvalFrameDefault ()
#48 0x000055db892d142c in _PyFunction_Vectorcall ()
#49 0x000055db892b9abb in _PyEval_EvalFrameDefault ()
#50 0x000055db892d142c in _PyFunction_Vectorcall ()
#51 0x000055db892b9abb in _PyEval_EvalFrameDefault ()
#52 0x000055db892d142c in _PyFunction_Vectorcall ()
#53 0x000055db892b9abb in _PyEval_EvalFrameDefault ()
#54 0x000055db892d142c in _PyFunction_Vectorcall ()
#55 0x000055db892b9abb in _PyEval_EvalFrameDefault ()
#56 0x000055db892d142c in _PyFunction_Vectorcall ()
#57 0x000055db892bf34a in _PyEval_EvalFrameDefault ()
#58 0x000055db892d142c in _PyFunction_Vectorcall ()
#59 0x000055db892b98cc in _PyEval_EvalFrameDefault ()
#60 0x000055db892b6016 in ?? ()
#61 0x000055db893ab8b6 in PyEval_EvalCode ()
--Type <RET> for more, q to quit, c to continue without paging--
#62 0x000055db893d6918 in ?? ()
#63 0x000055db893d01db in ?? ()
#64 0x000055db893d6665 in ?? ()
#65 0x000055db893d5b48 in _PyRun_SimpleFileObject ()
#66 0x000055db893d5793 in _PyRun_AnyFileObject ()
#67 0x000055db893c82ce in Py_RunMain ()
#68 0x000055db8939e70d in Py_BytesMain ()
#69 0x00007ff18d8d1d90 in __libc_start_call_main (main=main@entry=0x55db8939e6d0, argc=argc@entry=85, argv=argv@entry=0x7fff67751248) at ../sysdeps/nptl/libc_start_call_main.h:58
#70 0x00007ff18d8d1e40 in __libc_start_main_impl (main=0x55db8939e6d0, argc=85, argv=0x7fff67751248, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fff67751238) at ../csu/libc-start.c:392
#71 0x000055db8939e605 in _start ()

TheTechromancer avatar Oct 22 '24 20:10 TheTechromancer

Fixed in https://github.com/blacklanternsecurity/bbot/pull/1901.

TheTechromancer avatar Oct 31 '24 02:10 TheTechromancer