Jonas Spenger
Jonas Spenger
Connect PortalsJS (JS) with the Portals Core (JVM) runtime. This is required for #132.
It should be possible to checkpoint and recover to/from external storage. For example, we should implement some connector which allows us to checkpoint to and recovery from Amazon S3, or...
There should be a mechanism for persisting/checkpointing message queues to disk, as well as recovering message queues from disk. This is to enable consistently checkpointing and recovering the Portals interpreter....
The Portals runtime currently only supports in-memory state, and does not support checkpointing/recovering state. We should fix this, and implement a suitable persistent state backend. RocksDB is a good choice...
The final goal for the first release is to have an elastic, serverless Portals deployment. For the elastic part, the runtime executes on a Kubernetes cluster, and automatically makes decisions...
There should be a distributed execution mode for Portals. The workloads should be split into partitions, and executed across distributed nodes. The distributed execution should provide fault-tolerance. The nodes are...
Once the wanted features for the new statebackend, together with the checkpointing and recovery, has been implemented, tested, and cleaned up, we will merge it into main. This is an...
It would be good to have some ways for running tests on nightly builds. The solution should be very simple, and should not complicate the workflow. We should compare this...
One of the pervasive issues in Portals has to do with context functions. For example, this is the current implementation of the `MapTask` in Portals: ```scala private[portals] case class MapTask[T,...
In adding serialization we have encountered numerous errors with serialization. We should diagnose and deal with these errors. This is related to, and blocks the integration of, PR #130. Problems:...