storm icon indicating copy to clipboard operation
storm copied to clipboard

storm topology online update

Open xiaokang opened this issue 12 years ago • 0 comments

This PR is for #540. The core code modification is about 300 lines and many others are generated by thrift.

main points:

  1. interface changes (compatible with old versions) 1.1 zk add :topology-version and :update-duration-sec fields to StormBase:status map 1.2 zk add :topology-version to executor heartbeat 1.3 worker local state add versions dir to storm worker's current running topology-version 1.4 nimbus add updateTopology interface 1.5 add topology-version field to storm.thrift three struct: TopologySummary ExecutorSummary TopologyInfo
  2. update process 2.1 storm client run "storm jar xxxx -c topology.update=true" to invoke topology update process 2.2 storm client upload new jar file to nimbus 2.3 storm client call nimbus updateTopology interface 2.4 nimbus check the new topology and replace stormdist/storm-id dir 2.5 nimbus update StormBase in zk, set :topology-version(for destination version) and :update-duration-sec(for all workers update process duration) fields to StormBase:status map 2.6 supervisors check zk StormBase and do the update work if topology's local version is not the same with zk version 2.6.1 sync-supervisor download the latest code from nimbus to local stormdist/topology-version dir 2.6.2 each supervisor schedule the topology's worker update at a rand(expect-max-update-time) time point 2.6.3 sync-process check local worker version, if it is not the same with sync-supervisor downloaded version and update time point reached, set worker state to a new :update state 2.6.4 sync-process kill workers in :update state as normally 2.6.5 sync-process restart killed worker as normally, expect that read topology and conf from stormdist/topology-version dir 2.6.6 new worker heartbeat to zk with new topology-version, it can be displayed on web ui to check update progress

xiaokang avatar Nov 12 '13 17:11 xiaokang