simpleflow icon indicating copy to clipboard operation
simpleflow copied to clipboard

Local execution: serialize and deserialize objects before passing them to task

Open ggreg opened this issue 10 years ago • 3 comments

The local execution does not serialize and deserialize the objects it passes to the task handlers. However users expect the local execution to mimic distributed execution and use it to test a workflow will work. This implicit promise is broken when a workflow definition calls tasks with objects that does not serialize to Python.

ggreg avatar Nov 10 '14 09:11 ggreg

Good idea !

Maybe we could pickle it instead on json ?

ampelmann avatar Nov 10 '14 09:11 ampelmann

We should let the user select the appropriate serialization format for their usage. JSON is currently used because it is the default serialization format in SWF and it dumps most simple Python data structure.

Pickle might be a good choice to cover a broader range of objects and data structures.

ggreg avatar Nov 10 '14 17:11 ggreg

I think this depends heavily on what serialization SWF supports.

darkjh avatar Nov 10 '14 20:11 darkjh