xnmt
xnmt copied to clipboard
neulab
Customizable logging
It would be nice if one could customize the log message during training/evaluating, by e.g. providing the formatting string as an option in YAML. Ideally this would enable the user to specify
- Which metrics to display among a list of available and documented metrics (log likelihood, l2 regularization term, words/s vs sents/s, decimal epoch number vs percentage in current epoch...)
- The precision ('%.5f'/'%.1e'/...)
- For BLEU score the option of showing on the 0-100 scale
This would be super convenient.
Agree, and this would be quite easy to do:
- make DevLossTracker and TrainLossTracker Serializable
- pass them as arguments (with bare default values) to SimpleTrainingTask (for dev loss tracker) / to all TrainingRegimen implementations (for train loss tracker)
- add arguments to
__init__of both loss trackers to configure them as desired
@msperber I have a question regarding serialization in general:
say I so what you said (make DevLossTracker and TrainLossTracker Serializable), will this be backward compatible? As in will I be able to load models trained on eg the current master?
And more generally what kind of changes break backward compatibility?
It will be backward compatible as long as you specify default arguments. So basically when adding stuff, if your updated code can still parse the old model file, things should be fine. Removing or renaming arguments is more likely to break backward compatibility.