deepforge
deepforge copied to clipboard
Disconnect/reconnect to/from long-running interactive sessions
Especially given the upcoming interactive editor for training neural networks (#1816), it is likely that users will want to train long running jobs that they can later monitor and check in on. To that end, interactive sessions should be able to run without a connected browser connection and resume them at a later date.
This actually raises a bigger question about the current framework for interactive editors/sessions. Currently, the browser connects to the compute instance using DeepForge as a proxy (via the InteractiveCompute router). An alternative approach would be to add a server-side plugin as shown below (on the right) and to persist the data in the model from the start (probably with locks so only one user can open a node at a time):
In this case, the plugin would need to expose an editor-specific API to the browser and project edits would need to be sent to the plugin. The plugin would manage communication between the compute instance and the project. This would make reconnecting/suspending sessions quite simple as the plugin would remain running when the browser has been closed.
The biggest challenge of this approach lies in detecting abandoned sessions (so we don't have essentially a "session leak") and locking the nodes so they only have a single editor at a time.
It is also unclear if artifact explorers should behave the same way... It is more likely that they will not be quite as long running though it certainly is possible.