skorch icon indicating copy to clipboard operation
skorch copied to clipboard

Print param groups after initialization

Open ottonemo opened this issue 6 years ago • 3 comments

We should be verbose about the param groups that are added after initialization (since we support wildcards and it may be intransparent which param groups are created from that).

For example:

>>> net = Net(optimizer__param_groups=[('dense*', {'lr': 0.03})])
Initializing module!
Setting param group {'lr': 0.03} for dense0.weight, dense0.bias, dense1.weight, dense1.bias.

ottonemo avatar Jul 19 '18 11:07 ottonemo

I can see two ways to implement this feature:

  1. Use a callback with a sink parameter that prints the param group during on_train_begin.
  2. Print the param group in NeutralNet.initialize_optimizer when verbose is truthy.

What do you think?

thomasjpfan avatar Aug 09 '18 22:08 thomasjpfan

@ottonemo do you have an opinion on that?

benjamin-work avatar Aug 16 '18 12:08 benjamin-work

Since the introduction of virtual parameters we have dedicated setter methods for these parameters. This means that we have two places where this message may be relevant:

  1. during initialization from Net kwargs
  2. when setting using set_params(optimizer__0__lr=0.03) There should probably a utility function to print this message (of course we could think about a special callback that gets triggered when parameters are set but I don't think this is particularly helpful).

ottonemo avatar Dec 12 '18 13:12 ottonemo