composer
composer copied to clipboard
Refactor logging of `rank_zero_seed`
This line is logging the rank_zero_seed
as if it is time series data, but it's a static value that is only configured once. It looks very out of place in logger backends like WandB. I think rank_zero_seed
should be stored in config somewhere but not logged as time series.
This can be done by either promoting it to the TrainerHparams
, or creating a new update_config()
function that LoggerDestinations
have to implement.
Also a UX bug -- if each rank is logging in distributed training, then the seed
stored in each rank's config is now incorrect because Trainer
has overridden it to be rank_zero_seed + rank
. Are there any other codepaths we are overriding values in the config?
https://github.com/mosaicml/composer/blob/204129b810ce71d4089781b6ee2bef9f0b2703f1/composer/trainer/trainer.py#L873
This is what WandB looks like right now:
I like the idea of implementing update_config
in the logger destinations, as different loggers may want to store config differently.
Closed by PR #1416