Explanation to searching for a lot skip-connect

[Catosine]

Hello @AnnaAraslanova ! Thanks for your work! It's fantastic!

I've noticed that you mentioned that when searching on Cifar-10, your code got a structure with a lot of skip-connect. This is actually caused by "overfitting", for the gradient-based algorithm always prefers the fast gradient descent operations, which unfortunately, is the skip-connect here.

To deal with this issue, P-DARTS actually raises an interesting regularization to those skip-connects. You may check their codes at https://github.com/chenxin061/pdarts

You can also find their paper at https://arxiv.org/abs/1904.12760

GL,