machinery

machinery copied to clipboard

Worker # and worker concurrency is ambiguous

1

Currently there are multiple ways to set a limit on the number of concurrently of running tasks. 1. Run more workers asynchronously via `LaunchAsync` in which each worker will call...

ianlewis

Semaphore channels don't need to be initialized

2

Semaphore channels can be push style rather than pull. For example: https://github.com/RichardKnop/machinery/blob/7ccba3d50e4edd1489bd0ab1743534920deb093b/v2/brokers/redis/goredis.go#L97-L102 Can be implemented like: ```golang pool := make(chan struct{}, concurrency) select{ case pool

ianlewis

Fixes #737

ianlewis

Will there be a web UI for machinery like asynqmon => (https://github.com/hibiken/asynqmon) or flower => (https://flower.readthedocs.io/en/latest/) for python celery

encryptblockr

Potential new features and PRs

1

Hi @RichardKnop and all contributors to this awesome library! We are investigating the possibility to use machinery to power our task queue and it looks really great. Just one thing...

zxia-wish

Reads on maps in the eager backend are not protected from concurrent read & writes. Ex: https://github.com/RichardKnop/machinery/blob/master/v2/backends/eager/eager.go#L160

ianlewis

Results aren't expired from the eager backend so it can cause a memory leak if used for anything other than limited use.

ianlewis

Concurrency set on a worker passes concurrency to the broker implementation in `StartConsuming`. Each broker implementation returns an error to the worker if any concurrent task fails in which the...

ianlewis

it will loop call requeueMessage if broker appear a task that not registered

2

https://github.com/RichardKnop/machinery/blob/master/v1/brokers/redis/redis.go#L338

Hzyhan

Get connected workers

2

Is there a way to get all connected workers to the broker? I couldn't find anything in the API documentation.

taythebot