Shengquan Ni
Shengquan Ni
This PR includes a full lifecycle of log-based fault tolerance: 1. Added `LogStorage`, which is an abstraction of where the log is persisted. Currently implemented LocalFS and HDFS. 2. Added...
Right now we don't have a UI component for importing/exporting a workflow, which makes the workflow migration between different Texera services very painful.
Our engine creates an `OperatorExecutor` upon the creation of a worker actor. However, the `OperatorExecutor` is first created by calling a `(Int) => IOperatorExecutor` lambda function on the controller, then...
This PR finished recovery, with some refactoring changes for the DP Thread and network communication actor: 1. Added global recovery manager which manages the workflow recovery state. It notifies the...
Copied from #1589. This change also includes #1592.
This PR is mainly about adding a replay mechanism to both controller and worker but I also did the following along with the changes: 1. Refactored the logging part by...
Copied from #1606.
Copied from #1612.
It takes a long time to load a workflow because we send many EditingTimeCompilationRequest during the construction of the frontend workflow on the canvas.
In order to enhance the monitoring capabilities, it is essential to provide real-time visibility into the status of operators and the edges connecting them. Within our backend system, we have...