simpleflow
simpleflow copied to clipboard
Local execution: serialize and deserialize objects before passing them to task
The local execution does not serialize and deserialize the objects it passes to the task handlers. However users expect the local execution to mimic distributed execution and use it to test a workflow will work. This implicit promise is broken when a workflow definition calls tasks with objects that does not serialize to Python.
Good idea !
Maybe we could pickle
it instead on json
?
We should let the user select the appropriate serialization format for their usage. JSON is currently used because it is the default serialization format in SWF and it dumps most simple Python data structure.
Pickle might be a good choice to cover a broader range of objects and data structures.
I think this depends heavily on what serialization SWF supports.