Failure handling in plan execution

[wynot12]

In current AsyncDolphinPlanExecutor does not consider any failures in each step of reconfiguration. So if a failure happens PlanExecutor's behavior is undefined (e.g., deadlock or complete hiding fails).

To address it, we need to define and expose these failures, and react with proper policies.

We may start from following naive policies:

• Stop the execution and retry it several times.
• Stop the execution and just forget about this plan. #90 is a one of sub issues.