Independent, async node execution

[Keavon]

Support for long-running async nodes that execute sequentially, show progress, allow aborting/retrying, and permit graph modification while running

Also:

• Sandbox the graph executor in a web worker so it can recover from panics
• Sandbox individual nodes that panic so they can be disabled and let the remainder still execute

[0HyperCube]

We already have async nodes. However it is not trivial to communicate the state of the graph execution to the front end. How would you propose to do this?

[Keavon]

At the moment, while we do technically have async execution, it still blocks our event loop which requires a full start-to-finish graph compilation and execution before showing the result and returning from the event loop. (At least that's how I think it works currently? It seems to behave that way but I haven't rigorously traced the code.)

I'll have to defer to @TrueDoctor for his thoughts on how this should be implemented in our current architecture.

[TrueDoctor]

Nodes need to return some sort of meaningful data. What you are proposing is essentially polling a node for completion and returning some sort of default value until the polling is successful. This could just be implemented as a node. The more interesting issue is how we can express this in terms of the caching model. It could make sense to treat long-running tasks as external track their state there as a form of extracting the impure state.