marie-ai icon indicating copy to clipboard operation
marie-ai copied to clipboard

Implement work scheduling and distribution

Open gregbugaj opened this issue 2 years ago • 0 comments

Currently basic load balancing is used for work distributions, this needs to be replaces with much more robust solution. In original jina-ai work distribution is done via gRpc, this does work however it has number of limitations, this is where our implementation will differ significantly.

Objectives :

  • Predictable workflow (Directed acyclic graph)
  • Scheduling and Prioritization
  • Resilience to failure
  • Reliability

gregbugaj avatar Aug 17 '22 13:08 gregbugaj