Skip eval on train and test for self-reporting results

[Niccolo-Ajroldi]

Feature request: allow users to skip eval on train and test

Evaluating on the training and test sets is time-consuming and not necessary for self-reporting results. We should add a flag that allow the user to skip eval on these datasets, to make scoring faster.

Accordingly, in this scenario we should modify:

goals_reached = (
              train_state['validation_goal_reached'] and
              train_state['test_goal_reached'])

into:

goals_reached = (train_state['validation_goal_reached'])

This would speed up self-evalution even more, by stopping training when validation target is reached, avoiding unnecessary usage of computational resources.

[priyakasimbeg]

You're right this is a good suggestion to allow to skip on train and test splits.

[fsschneider]

We plan to discuss feature requests like these in the benchmark code during the WG meeting on Thursday, 9/5.