What experiment tracker is a good choice?

[KonstantinUshenin]

Hello! I use Haiku+Jax for research. I have written a class that tracks my experiments in a simple folder structure. It is helpful for 10-20 runs. However, I want to use more powerful tools to track 100-1000 experiments and gather results from many computational nodes together.

What is a good experiment tracker for Haiku? I think PyMLPipe or DVCLive is a good choice, but I cannot make a choice. Is there something better?

[nlsfnr]

I personally use Weights and Biases. They have remote logging via their website and explicitly focus on parameter sweeps etc. AFAIK they also have an option to host the server for the logging yourself.

[joglekara]

i use mlflow because

• i find its abstractions very useful as a general numerical experiment manager (deep learning, PDE simulations, coupled ODE solve etc.).
• it's self hosted and can be run locally or on the cloud w/ a dedicated DB and object store (just like wandb).
• it is open source