bhack

> > I think that we could separate Shuffle in Shufflenet and KLP layers as they have two very different behaviors. > > > > Can we maintain the original...

> Can we clarify what the differences are? Two points: - I've [linked above](https://github.com/tensorpack/tensorpack/blob/master/examples/ImageNetModels/shufflenet.py#L40-L49) a quite popular TF implementation at that time and If I remember correctly from the paper...

> I think the KPL layers may contain differentiable layers in the future. Isn't this point partially the root cause of this PR? If we are going in this direction...

> From what I can tell I think ShuffleNet is a super rare one off. Also If we consider this a quite rare case (I am not so future proof...

> Curious, will keras_cv always provide only the random transformation layer? I think this part of the topic was early covered at https://github.com/keras-team/keras-cv/pull/122#discussion_r803214729. IMHO from that discussion we have not...

> Ok, yeah if there is no random behavior in one case and it is fully random in the other it should really be a different layer. I misunderstood the...

I suggest to take a look at [Continual Learning on the Edge with TensorFlow Lite](https://arxiv.org/abs/2105.01946) And https://arxiv.org/abs/2105.13127

/cc @vlomonaco

Another interesting scenario to evaluate is training in the context of Edge federated learning: https://github.com/tensorflow/federated/issues/749 https://arxiv.org/abs/2104.03042 https://arxiv.org/abs/1909.11875 https://www.sciencedirect.com/science/article/pii/S266729522100009X

/cc @gdemos01 @akhilmathurs