Learning rates and HyperParameters

[Cmeo97]

Hi!

I am trying to replicate the plots you show, but I'm not able to find Learning Rates and HyperParameters on the json file created. Where can I find them?

Also, I was wondering if I can find the descriptions of the quantities somewhere in the documentation. I found a table with names and descriptions, but for example the dimensions are not explained and for some quantities I am not able to understand why they have such dimensions on the json file.

Thanks a lot!

Bests, Cristian

[fsschneider]

Hi Cristian,

to reproduce our results, please take a look at https://github.com/fsschneider/cockpit-experiments This repository should hold all the information and code to replicate our experiments. Let me use experiment 07_learning_rate_selection (shown as Figure 5 in the paper) as an example:

https://github.com/fsschneider/cockpit-experiments/tree/main/experiments/07_learning_rate_selection

The four run scripts run_*.py produce the raw data. In each, we specify the list of learning rates to use, e.g. https://github.com/fsschneider/cockpit-experiments/blob/a9eaf3dc5da5a58356ac0eef25a11235bf0891c4/experiments/07_learning_rate_selection/run_cifar10_3c3d.py#L6

Our raw results are also available via the results.zip. You can then use the plotting scripts to create the figure shown in the paper. You could either use our raw data, using the results.zip or recreate the data yourself by using the run scripts.

Looking at the results.zip you can also see that the json files in it contain the hyperparameters such as the learning rate.

Regarding your second question:

Our documentation lists all the quantities included in Cockpit, see here: https://cockpit.readthedocs.io/en/latest/api/quantities.html. In the paper, https://arxiv.org/pdf/2102.06604.pdf, you can also find an overview table linking to a mathematical description of most quantities in the appendix in Table 2. The entire Section C in the appendix is dedicated to describing the quantities.

If you don't understand the dimensions of certain quantities, feel free to ask me about them directly.

Hope that this helps, Frank

[fsschneider]

I am closing this issue as it is stale. Please feel free to open it again if you have further questions.