mala icon indicating copy to clipboard operation
mala copied to clipboard

Zero validation data loss during hyperparameter optimization

Open nerkulec opened this issue 1 year ago • 1 comments

When running TPE hyperparameter optimization with

parameters.hyperparameters.hyper_opt_method = "optuna"
parameters.running.after_before_training_metric = "band_energy"
...

validation data loss (during training) is always 0

Epoch 0: validation data loss: 0.000e+00, training data loss: 8.739e-07
Time for epoch[s]: 1121.9449763298035
training time: 744.3423397541046
Epoch 1: validation data loss: 0.000e+00, training data loss: 4.167e-07
Validation accuracy has not improved enough.
Time for epoch[s]: 1127.3465538024902
training time: 744.2401514053345
Epoch 2: validation data loss: 0.000e+00, training data loss: 3.374e-07
Validation accuracy has not improved enough.
...

This leads to always terminating the training after early_stopping_epochs as the validation loss is not improving.

nerkulec avatar Nov 03 '23 13:11 nerkulec