Load distribution between nodes.

[morlz]

I think it is worth sorting nodes by flops and executing more layers on those nodes where there is more performance. This will speed up execution in cases where the model size is smaller than the cluster memory. It is necessary to pass the size of the current model to PartitioningStrategy for this to work.

[morlz]

I can try to implement this

[AlexCheema]

This will be the default in exo v2!

[rltakashige]

A version of this is in EXO 1.0 - plan to improve this though.

[rltakashige]

Please open again if this is still an issue!