django-rq icon indicating copy to clipboard operation
django-rq copied to clipboard

job_timeout is not killing subprocess

Open nadermx opened this issue 10 months ago • 0 comments

I currently have a django route that enqueue something like so

            convert = queue.enqueue(
                to_epub,
                args=(f.inputfile, outputfile, postdata),
                job_timeout=config.JOB_TIMEOUT
            )

And the ebook-pdf function looks like so

def to_epub(filename, outfile, options):
    _, file_extension = os.path.splitext("%s" % filename)
    if file_extension == '.doc':
        temp_file = "%s" % filename
        temp_file = temp_file.replace('.doc', '.docx')
        doc_docx(filename, temp_file, options)
        filename = temp_file

    cmd = 'ebook-convert "%s" "%s"' % (filename, outfile)
    process = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)


    try:
        ConversionFile.set_to_finished(options.get('uuid'))

        if process.stderr and "error" in process.stderr.decode("utf-8"):
            return {'error': process.stderr.decode("utf-8")}
        return [outfile]
    except Exception as e:
        print(str(e))

Yet for some reason, some times it does not kills the process after the job_timeout which causes at some point when over 50 of these hang for the queue to stop processing jobs.

Using the most recent version of django-rq and rq

nadermx avatar Sep 09 '23 00:09 nadermx