confusing/nonintuitive variable naming conventions

[vagarwal87]

I am getting somewhat confusing results in the variable names of the best model. Here is my code:

batchsize = {{choice([32, 64, 128, 256])}}
activationFxn = {{choice(['relu', 'elu', 'selu', 'tanh'])}}
numconvlayers = {{choice(['one','two','three','four'])}}
numconvlayers = conditional(numconvlayers)
numFiltersConv1 = {{choice([16, 32, 64, 128])}}
FilterLenConv1 = {{choice([2, 4, 8, 16])}}
MaxPool1 = {{choice([2, 4, 8, 16])}}
numFiltersConv2 = {{choice([16, 32, 64, 128])}}
FilterLenConv2 = {{choice([2, 4, 8, 16])}}
MaxPool2 = {{choice([2, 4, 8])}}
numFiltersConv3 = {{choice([16, 32, 64, 128])}}
FilterLenConv3 = {{choice([2, 4, 8, 16])}}
MaxPool3 = {{choice([2, 4])}}
numFiltersConv4 = {{choice([16, 32, 64, 128])}}
FilterLenConv4 = {{choice([2, 4, 8, 16])}}
numdenselayers = {{choice(['one','two'])}}
numdenselayers = conditional(numdenselayers)
Dense1 = {{choice([16, 32, 64, 128])}}
Dropout1 = {{uniform(0, 1)}}
Dense2 = {{choice([16, 32, 64, 128])}}
Dropout2 = {{uniform(0, 1)}}
auxdata = Input(shape=(X_trainaux.shape[1:]), name='aux')
input = Input(shape=X_train.shape[1:], name='inputseq')

x = Conv1D(numFiltersConv1, FilterLenConv1, padding='same', kernel_initializer='glorot_normal', input_shape=X_train.shape[1:],activation=activationFxn)(input)
x = MaxPooling1D(MaxPool1)(x)

if numconvlayers != 'one':
    x = Conv1D(numFiltersConv2, FilterLenConv2, padding='same', kernel_initializer='glorot_normal',activation=activationFxn)(x)
    x = MaxPooling1D(MaxPool2)(x)
    if numconvlayers != 'one' or numconvlayers != 'two':
        x = Conv1D(numFiltersConv3, FilterLenConv3, padding='same', kernel_initializer='glorot_normal',activation=activationFxn)(x)
        x = MaxPooling1D(MaxPool3)(x)
        if numconvlayers == 'four':
            x = Conv1D(numFiltersConv4, FilterLenConv4, padding='same', kernel_initializer='glorot_normal',activation=activationFxn)(x)

x = Flatten()(x)
x = merge([x, auxdata], mode='concat', concat_axis=1)
x = Dense(Dense1)(x)
x = Activation(activationFxn)(x)
x = Dropout(Dropout1)(x)
if numdenselayers == 'two':
    x = Dense(Dense2)(x)
    x = Activation(activationFxn)(x)
    x = Dropout(Dropout2)(x)
main_output = Dense(1)(x)
model = Model(input=[input, auxdata], output=[main_output])
model.compile(Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0),'mean_squared_error', metrics=['mean_squared_error'])
result = model.fit([X_train, X_trainaux], y_train, batch_size=batchsize, shuffle="batch", epochs=2, validation_data=[[X_valid, X_validaux], y_valid]) #, callbacks=[earlystop_cb]
mse = result.history['val_mean_squared_error'][-1]
parameters = space
parameters["loss"] = mse
print(parameters)
return {'loss': mse, 'status': STATUS_OK, 'model': model}

Best results are the following: {'numdenselayers': 'two', 'loss': 0.50110858917236323, 'batchsize': 32, 'Dropout1_1': 0.17311452951878298, 'numconvlayers': 'three', 'FilterLenConv1_1': 8, 'Dropout1': 0.14597365443010546, 'numFiltersConv1': 128, 'activationFxn': 'elu', 'numFiltersConv1_1': 64, 'numFiltersConv1_2': 16, 'numFiltersConv1_3': 64, 'numFiltersConv1_4': 16, 'numFiltersConv1_5': 128, 'FilterLenConv1_3': 8, 'FilterLenConv1_2': 8, 'FilterLenConv1_4': 4, 'FilterLenConv1': 4, 'MaxPool2': 8, 'MaxPool3': 4}

What is confusing is that there is no MaxPool1, Dense1, or Dense2 values which I defined, and it adds a number of new names with underscores such as "numFiltersConv1_2" instead of the names I assigned. Why is this, and is there a way to simply use the names I defined? In other runs, I get back 2 convolutional layers are the best, yet it still returns MaxPool3, which should only be defined if a 3rd convolutional layer was chosen. I don't see an explanation for this behavior in the documentation or how to properly use unique names that are easily interpretable. If I don't use any names the results get even more jumbled up in terms of being able to match the best results with the corresponding hyperparameters.

Any help is appreciated!

[mingfeisun]

You can find out the corresponding parameter positions in your network by checking the following part at the beginning of your training:

Resulting replaced keras model: 1: def keras_fmin_fnct(space): 2: ....x = MaxPooling1D(space['Dropout')(x)

Just pay attention to those space['...' variables. Then you can easily find the position correspondences.