Reinitialize weights of neural net

[ikki407]

In Keras-implemented neural net, to avoid recompile, initial weights after compilation is saved and used at the next beginning of training in cross validation. However, the initial weights are same in all fold-training. So initial weights should be changed at each training.

A possible solution is passing the argument of compilation (e.g., optimizer, loss, and metrics). In binary_class.py,

class ModelV2(BaseModel):
        def build_model(self):
            model = Sequential()
            model.add(Dense(64, input_shape=nn_input_dim_NN, init='he_normal'))
            model.add(LeakyReLU(alpha=.00001))
            model.add(Dropout(0.5))

            model.add(Dense(output_dim, init='he_normal'))
            model.add(Activation('softmax'))
            sgd = SGD(lr=0.1, decay=1e-5, momentum=0.9, nesterov=True)

            compile_options = {
                                           'optimizer': sgd, 
                                           'loss': 'categorical_crossentropy', 
                                           'metrics': ["accuracy"]
                                           }

            return KerasClassifier(nn=model, compile_options=compile_options, **self.params)

In base.py,

import copy

class KerasClassifier(BaseEstimator, ClassifierMixin):
    def __init__(self,nn):
        self.nn = nn

    def fit(self, X, y, X_test=None, y_test=None):
        self.compiled_nn = copy.copy(self.nn)
        self.compiled_nn.compile(**self.compile_options)

        return self.compiled_nn(X, y)

But this approach leads memory consumption...