storm icon indicating copy to clipboard operation
storm copied to clipboard

make scheduling evenly distribute slots across all hosts

Open lyogavin opened this issue 12 years ago • 1 comments

#359

Changed the way even schedule distribute slots into also considering the number of used slots across all hosts. Make sure the new assignment use slots evenly across all hosts.

lyogavin avatar Apr 12 '13 01:04 lyogavin

Just change sort-slots works on out cluster. The code is bellow:

Index: src/clj/backtype/storm/scheduler/EvenScheduler.clj
===================================================================
--- src/clj/backtype/storm/scheduler/EvenScheduler.clj
+++ src/clj/backtype/storm/scheduler/EvenScheduler.clj
@@ -7,7 +7,12 @@
 :implements [backtype.storm.scheduler.IScheduler]))

 (defn sort-slots [all-slots]
-  (let [split-up (vals (group-by first all-slots))]
+  (let [split-up 
+         (map second 
+           (reverse 
+             (sort 
+               (for [[host ports] (group-by first all-slots)] 
+                 [(count ports) ports]))))]
     (apply interleave-all split-up)
     ))

xiaokang avatar Apr 16 '13 04:04 xiaokang