storm icon indicating copy to clipboard operation
storm copied to clipboard

fixes #551, when rebalance ,remove out of date assignment

Open vinceyang opened this issue 11 years ago • 4 comments

to fixes #551 ,when rebalance the rebalance topolgy's assignment has out of date ,must clear these assignment from zookeeper,.

vinceyang avatar Sep 17 '13 12:09 vinceyang

This does seem like it will fix the re-balancing issue, but it does not address other schedulers that might take all resources from a previously running topology.

revans2 avatar Sep 18 '13 17:09 revans2

My reference was a mistake. Sorry for that.

herberteuler avatar Sep 19 '13 02:09 herberteuler

@revans2 , scheduler address all resources from a previously running topology,.seems we have to mark all topology's a flag to describe does topology in effect. supervisor Ignore the renbalancing topology's assigment. do you have any idea?

vinceyang avatar Sep 22 '13 01:09 vinceyang

@vinceyang I am not really sure what you mean by that. The scheduler looks at the cluster as a whole and all topologies that have been requested to run on it. It has the power to remove or change assignments for any topology as it sees fit. I found that when a scheduler removes all of the assignments for a previously running topology, the code that decides if it should update ZK makes a mistake and does not update ZK.

I simply pointed out that the current patch seems to fix the case for a rebalance that can cause a scheduler to remove the resources for a previously running topology, but it does not address the general case where a scheduler might do the same thing for a different reason. Assume that there is a priority queue and a higher priority topology comes in so the lowest priority topology is stopped.

revans2 avatar Sep 23 '13 19:09 revans2