TileDB
TileDB copied to clipboard
Adds functionality for creating and executing a task graph
This PR adds functionality for creating and executing a task graph. The functionality allows for tasks to dynamically generate sub task graphs while the graph is being executed.
All internal algorithms (reads, writes and the upcoming advanced computations) will be refactored to use the task graph functionality. The task graph stores logs and statistics that will make it very easy to fix both bugs and performance issues.
Minimal example:
// Create task graph
auto task_graph = tdb_make_shared(TaskGraph);
auto task0 = task_graph->emplace([]() {return Status::Ok();}, "0");
auto task1 = task_graph->emplace([]() {return Status::Ok();}, "1");
auto task2 = task_graph->emplace([]() {return Status::Ok();}, "2");
task_graph->succeeds(task2, task0, task1);
// Execute the task graph
ThreadPool tp;
tp.init(std::thread::hardware_concurrency());
TaskGraphExecutor tge(&tp, task_graph);
tge.execute(); // This is non-blocking
tge.wait(); // This is blocking
TYPE: FEATURE DESC: Adds functionality for creating and executing a task graph.
Closing in favor of #3328 #3366 #3453 and others in the series.