biothings.api icon indicating copy to clipboard operation
biothings.api copied to clipboard

Consider separating scheduling as an individual feature

Open namespacestd0 opened this issue 3 years ago • 1 comments

Currently, many actions that trigger subsequent actions define these automations within the preceding action. This pattern sometimes breaks the feature isolations, and make automation management spread out across the application. When encountering errors, it can also be difficult to define the retry strategies because of the lack of a centralized component that tracks the scheduling, and execution of tasks. The stated issues come from real-world use cases, but are also rudimentary, and deserve more deliberation of implementation details.

namespacestd0 avatar Jul 22 '21 03:07 namespacestd0

Typically a message queue is used to schedule things like these. RabbitMQ, ZeroMQ, or Apache Kafka (technically not just a message queue).

Using a message queue can greatly decouple the components and can be very helpful in many ways down the line, but such architectural change is pretty complicated.

zcqian avatar Jul 22 '21 04:07 zcqian