aim

aim copied to clipboard

🚀 Feature

Support easy ways to collapse/hide nested run params

Motivation

Deep nested run params can occupy a large number of columns, it would be nice to have an easy way to collapse/hide all nested values beyond a certain depth, or below certain keys like the run params tab in single run page.

Pitch

In my case, my training config is a large nested json object containing hyperparameters, model architecture specifications, and configurations for distributed training. I'd like to upload this nested json object as run params to aim, but I'm worried that it will occupy too many columns which may potentially slow down the UI and/or make it hard for other people to find columns they need. It would be nice if we can easily collapse/hide deep nested run params, and even set something like a depth limit beyond which all nested run params are collapsed by default.

Alternatives

Happy to discuss alternatives

Hey @jiyuanq. That is a great point! We have been thinking about adding a project-level settings page, where users will be able to whitelist/blacklist those configs, which aren't necessarily needed to be displayed in tables. Basically the idea is to enable:

• whitelisting those config which should always be visible in tables
• blacklisting configs which will be hidden by default
• diff of the rest of configs - dynamically calculating and displaying those columns where at least 2 different values exist

Does this make sense to you?

Also if it is possible could you share how you use the table? Some use cases?

Hey @jiyuanq. That is a great point! We have been thinking about adding a project-level settings page, where users will be able to whitelist/blacklist those configs, which aren't necessarily needed to be displayed in tables. Basically the idea is to enable:

• whitelisting those config which should always be visible in tables
• blacklisting configs which will be hidden by default
• diff of the rest of configs - dynamically calculating and displaying those columns where at least 2 different values exist

Does this make sense to you?

Thank you for the reply! Yeah this makes a lot of sense to me.

One possible aim.toml (or pyproject.toml?!) config file layout:

[aim.ui.runs_explorer]
columns = [
  { keep = False },  # blacklist EVERYTHING
  { keep = True, selector = "[non_duplicate]" },
  { keep = True, tag = "loss" },
  { keep = False, tag = "loss", context = { scope = "train", ... } },
  { keep = True, tag = "some_whitelisted_metric" },
]

Alternatively, a .gitignore style string would also make sense (though one would have to be careful not to introduce edge cases in syntax, and inventing a new syntax is also harder to learn):

[aim.ui.runs_explorer]
columns_ignore = '''
*
??? idk
!loss
loss | context={scope="train"}
!some_whitelisted_metric
'''

@YodaEmbedding looks great! We are thinking about adding a more user friendly interface on the first place (settings page on the UI). But the source of truth will be a dotfile (like ones you shared), which will enable:

• more advanced configurations
• easy transferring configs to other projects
• sharing best practices with the community

I will make sure to keep you in the loop, as I noticed you have a pretty good experience in it (dotfiles). I believe your contribution here would be very essential. 🙌