Model checkpoint naming: create naming convention

[remtav]

Model checkpoint naming convention

A naming convention for models' weights checkpoints written during training has become essential for:

• quickly understanding what flavor of model checkpoint we are dealing with as checkpoint files are being moved across the universe;
• overiding parameters during inference to instantiate the right model architecture, with correct number of input and output channels;

As far as GDL is concerned, all other relevant information for reproducibility (ex.: git version) is saved inside checkpoint under "params" key, which essentially contains all config parameters used during training.

Proposed convention

[architecture-id]_[input bands]_[output classes]_[date].pth.tar i.e. [architecture-id]_[modalities-by-common-name]_[class1-id]-[classe2-id]-[classn-id]_[YYYYMMDD].pth.tar

Examples

• unet-smp_red-green-blue-nir_WAER-FORE-ROAI-BUIL_20220323.pth.tar
• unet-gdl_red-green-blue_WAER_20220323.pth.tar

Alternative with single-letter bands

unet-smp_RGBN_WAER-FORE-ROAI-BUIL_20220323.pth.tar

Implications

• GDL's model names (ie name of config file for each model, without .yaml extension) should be as static as possible since they'll serve as architecture id
• GDL should check that inputted modalities for imagery bands follow STAC common names
• If a model predicts 52 classes, for example, the checkpoint's name will be darn long! How should we prevent names from having >260 characters?
• If a same architecture is used twice for the same band and class combination, this convention will not differenciate them. This means, it won't be easy to determine, for example, what scores each model obtained on a particular dataset (like out internal benchmark dataset). Should our naming convention include a date or something that adds a bit more "uniqueness"?

Options for uniqueness

• Date: format? YYMMDD? iso8601? 20220323
• Random name (e.g. docker containers)?
• Hex code

Options for abbreviating name

• Class id as single letter
• Class id as integer between 0-255