burr icon indicating copy to clipboard operation
burr copied to clipboard

Umbrella issue -- parallelism improvements

Open elijahbenizzy opened this issue 1 year ago • 0 comments

All the improvements to parallelism that we want to make post the first release:

  • [ ] UI improvements
    • [x] Expand parallel apps inline (#421)
      • [x] Show success/failure
    • [ ] Ensure insights tab shows aggregation of all parallel sub-applications
    • [ ] Show the sub-applications graphs in the graph
    • [x] Ensure live mode works (#421)
    • [x] Ensure the hierarchy of traces is clear (#421)
  • [ ] Task creation/management
    • [ ] Add API for task IDs
      • [ ] Add API
      • [ ] Wire through to UI
      • [ ] Add documentation
  • [ ] Guard-rails around async problems
    • [ ] Ensure that generators that are both async and not async work as expected
  • [x] Wiring through persisters
    • [x] Allow wiring through of persisters to sub-trackers
      • [x] Implement this spec: https://burr.dagworks.io/concepts/parallelism/#persistence-tracking -- not currently doable
  • [ ] Execution improvements
    • [ ] Reevaluate execution API
    • [ ] Add guard-rails for executors (E.G. with async, etc...)
    • [ ] Allow customization of execution capabilities on a per-task basis
  • [x] Ensure recursion works -- multiple levels deep

elijahbenizzy avatar Nov 19 '24 19:11 elijahbenizzy