Sequoia
Sequoia copied to clipboard
"proper" logging to wandb
Need to make sure that the components of the repo that explicitly log stuff to wandb (BaselineMethod
) as well as the components that could be logged to wandb, (TestEnvironments
(gym Monitors), the Results
objects, etc) generate a clean, crisp, minimal summary of the performance of a given method, that can ideally be already clearly understandable on wandb without needing to customize / filter through a bunch of stuff.
There are basically two ways I know of to use wandb:
- Log everything, and clean it up later
- Keep it simple, only log what you need.
While I'd naturally be inclined to opt for the first option, I think it might be best to stick to something like the second option, unless something like the --versose
flag is passed, in which case we could just log everything.
I don't know all the tips and tricks of wandb yet, so I'd welcome any suggestions of things / plots / stuff we could generate "for free" for the users of the repo.