skorch
skorch copied to clipboard
Print param groups after initialization
We should be verbose about the param groups that are added after initialization (since we support wildcards and it may be intransparent which param groups are created from that).
For example:
>>> net = Net(optimizer__param_groups=[('dense*', {'lr': 0.03})])
Initializing module!
Setting param group {'lr': 0.03} for dense0.weight, dense0.bias, dense1.weight, dense1.bias.
I can see two ways to implement this feature:
- Use a callback with a
sink
parameter that prints the param group duringon_train_begin
. - Print the param group in
NeutralNet.initialize_optimizer
whenverbose
is truthy.
What do you think?
@ottonemo do you have an opinion on that?
Since the introduction of virtual parameters we have dedicated setter methods for these parameters. This means that we have two places where this message may be relevant:
- during initialization from
Net
kwargs - when setting using
set_params(optimizer__0__lr=0.03)
There should probably a utility function to print this message (of course we could think about a special callback that gets triggered when parameters are set but I don't think this is particularly helpful).