Flask-RQ2 icon indicating copy to clipboard operation
Flask-RQ2 copied to clipboard

Support passing arguments to worker constructor

Open shouichi opened this issue 6 years ago • 3 comments
trafficstars

Though rq.worker.Worker constructor accepts many arguments (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L159 ), it seems that Flask-RQ2 does not support passing arguments to worker constructor. Do you guys want to support this feature?

Background

rq.worker.Worker uses hostname and pid for its name by default (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L239) but we run workers in container and hostname/pid are all the same. As a result, workers failed to start due to the duplicated name (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L275).

Thanks!

shouichi avatar Nov 29 '18 02:11 shouichi

Assuming you mean the RQ.get_worker method: yes. But we shouldn't simply pass all **kwargs to Worker but define some options that are shared in the RQ instance instead. In other words let's try to keep Flask-RQ2 a high-level API that hides some of the low-level features.

Here are the arguments that need to be added:

  • [ ] name (add as RQ.worker_name?)
  • [ ] default_result_ttl (reuse RQ.default_result_ttl)
  • [ ] default_worker_ttl (add as RQ.default_worker_ttl)
  • [ ] job_monitoring_interval (add as RQ.job_monitoring_interval, and add as option to worker CLI callback)

jezdez avatar Nov 29 '18 16:11 jezdez

But we shouldn't simply pass all **kwargs to Worker but define some options that are shared in the RQ instance instead. In other words let's try to keep Flask-RQ2 a high-level API that hides some of the low-level features.

Got it (I understand the RQ's goal is keeping it simple).

  • [ ] name (add as RQ.worker_name?)

Make the default value None and allow users to pass arbitrary string. That why the RQ keeps the current behavior. Correct?

  • [ ] job_monitoring_interval (add as RQ.job_monitoring_interval, and add as option to worker CLI callback)

Do you mean adding an option (@click.option) to this func? https://github.com/rq/Flask-RQ2/blob/94f8d2f20bc2450fab8a6996ea82424deff3dd11/src/flask_rq2/cli.py#L136

Thanks!

shouichi avatar Nov 30 '18 00:11 shouichi

From my use of Flask-RQ2, I'd like to add: date_format log_format

These 2 args are useful for the logging

alvin1 avatar Dec 06 '18 03:12 alvin1