How to see the training history of the best model returned by hyperas?

[Crista23]

I would like to plot the training and validation loss and accuracy of the best performing model returned by hyperas. How can I see the training history of the best model? Thank you!

[jntorres]

You would simply change the way that you write your model function and initate Trials() before you call optim.minimize. Here is a short example, instead of

model.fit(X_train, Y_train, batch_size={{choice([64, 128, 25])}}, epochs={{choice([10, 15, 25])}}, verbose=2, validation_data=( X_valid, Y_valid), callbacks=[reduce_lr,early_stopping, checkpointer])

Save the output of your model to a variable, in this case history

history = model.fit(X_train, Y_train, batch_size={{choice([64, 128, 25])}}, epochs={{choice([10, 15, 25])}}, verbose=2, validation_data=( X_valid, Y_valid), callbacks=[reduce_lr,early_stopping, checkpointer])

in your return you would simply add

return {'loss': score, 'status': STATUS_OK, 
    'model': model,  'history.val_loss':history.history['val_loss'], 'history.val_acc': history.history['val_acc'],
    'history.loss': history.history['loss'], 'history.acc': history.history['acc'],
    'history.lr': history.history['lr'])

Then use trials.results to access that information

[harshilpatel312]

You can use tensorboard callback too..

Make sure you have loss and accuracy in metrics in model.compile. And add the following lines to your def create_model():

tensorboard = TensorBoard(log_dir="logs/model@{}".format(int(time.time())))
model.fit(x_train, y_train,
              batch_size={{choice([1024, 2048])}},
              epochs=10,
              callbacks=[tensorboard],
              verbose=1,
              validation_split=0.1)

And then run tensorboard --logdir=logs/ in Terminal and go to the link generated.

[Idakwo]

You would simply change the way that you write your model function and initate Trials() before you call optim.minimize. Here is a short example, instead of

model.fit(X_train, Y_train, batch_size={{choice([64, 128, 25])}}, epochs={{choice([10, 15, 25])}}, verbose=2, validation_data=( X_valid, Y_valid), callbacks=[reduce_lr,early_stopping, checkpointer])

Save the output of your model to a variable, in this case history

history = model.fit(X_train, Y_train, batch_size={{choice([64, 128, 25])}}, epochs={{choice([10, 15, 25])}}, verbose=2, validation_data=( X_valid, Y_valid), callbacks=[reduce_lr,early_stopping, checkpointer])

in your return you would simply add

return {'loss': score, 'status': STATUS_OK, 
    'model': model,  'history.val_loss':history.history['val_loss'], 'history.val_acc': history.history['val_acc'],
    'history.loss': history.history['loss'], 'history.acc': history.history['acc'],
    'history.lr': history.history['lr'])

Then use trials.results to access that information

Hi @jntorres This returns the history for all the models? Any idea how I can get for only the best model? Thanks

[Idakwo]

I would like to plot the training and validation loss and accuracy of the best performing model returned by hyperas. How can I see the training history of the best model? Thank you!

@Crista23 Hi, Did you get around this problem?

[misbahch6]

@Crista23 Did you find a way to access the training history of the best model ?