biothings.api
biothings.api copied to clipboard
Consider separating scheduling as an individual feature
Currently, many actions that trigger subsequent actions define these automations within the preceding action. This pattern sometimes breaks the feature isolations, and make automation management spread out across the application. When encountering errors, it can also be difficult to define the retry strategies because of the lack of a centralized component that tracks the scheduling, and execution of tasks. The stated issues come from real-world use cases, but are also rudimentary, and deserve more deliberation of implementation details.
Typically a message queue is used to schedule things like these. RabbitMQ, ZeroMQ, or Apache Kafka (technically not just a message queue).
Using a message queue can greatly decouple the components and can be very helpful in many ways down the line, but such architectural change is pretty complicated.