bbot icon indicating copy to clipboard operation
bbot copied to clipboard

Lots of "Task was destroyed but it is pending!" errors - possible ungraceful shutdown

Open mukesh-dream11 opened this issue 6 months ago • 1 comments

Describe the bug I am running bbot from a celery worker. This worker listens for tasks scheduled from my Django webserver, imports bbot and executes it like so:

from celery import shared_task

from logging import DEBUG, getLogger

from .models import Target, Scan, ScanStatus

logger = getLogger(__name__)


@shared_task
def scan_target(target_id):
    from bbot import Scanner, Preset

    getLogger("bbot").setLevel(DEBUG)

    target = Target.objects.get(id=target_id)
    logger.info(f"Scanning target: {target.url}")
    preset = Preset(
        target.url,
        presets=["kitchen-sink"],
        exclude_modules=[
            # ... snipped for brevity ...
        ],
    ).bake()
    scanner = Scanner(preset=preset)
    for event in scanner.start():
        Scan.objects.create(target=target, event=event.json())
    target.scan_status = ScanStatus.COMPLETED
    target.save()

After the scans are finished as per the logs emitted by bbot, sending Ctrl + C to my celery worker throws a bunch of errors. (Logs attached).

While these errors are not a showstopper (scan works and results are recorded just fine), I would like to keep my logs clean. I'm interested in using bbot in a production setting, so I would like to avoid fears of memory leaks, unfinished coroutines or messy cleanup.

BBOT Command Executed from Python as described above

OS, BBOT Installation Method + Version macOS 15.5 Sequoia, M4 Max, bbot version dev (commit: 901a19894fca2d5fbdf59a105ae12d6e28eb0909)

BBOT Config As presented above.

Logs

queuing.txt finished.txt errors.txt

Note that queuing.txt contains log entries for the framework queuing FINISHED messages to all the modules so that they can gracefully shut down. finished.txt indicates the individual modules receiving the FINISHED event. errors.txt shows a list of all modules that throw the Task was destroyed but it is pending! errors.

I may or may not be picking this up as a work item and submitting a PR to fix this issue, if I'm able to get to the bottom of this and fix it.

mukesh-dream11 avatar Jun 21 '25 23:06 mukesh-dream11

A sample full run with only one module (and dependencies) enabled:

full_logs.txt

mukesh-dream11 avatar Jun 22 '25 00:06 mukesh-dream11