[FR] Better visualization of nested params

[johny-c]

Proposal Summary

In my work, I compare runs with large and nested configurations stored as python dictionaries. This results in the Run comparison page looking like this:

As shown, the parameters that are themselves dictionaries are displayed in a single row of the table. I'm guessing this is because they are retrieved as plain strings when the page is loaded.

The visualization would be much more useful if each level of a nested configuration was displayed indented and in a new row, much like in a .yaml file. Maybe this can be done with a (recurrent) check if a param can be read as dictionary.

In case of deep nesting, there could be a fixed upper limit (e.g. 3) supported by mlflow. Ideally there would be a button in the UI, so the user can control the depth of nested params they want to see by expanding/contracting these params.

Moreover, since nested are not read as dictionaries, they are not searchable in order to filter runs.

What component(s), interfaces, languages, and integrations does this feature affect?

Components

• [x] area/tracking: Tracking Service, tracking client APIs, autologging

Interfaces

• [x] area/uiux: Front-end, user experience, JavaScript, plotting

[smurching]

Hi @johny-c, thanks for filing this! Do you need to log the params as a dictionary, or would it be feasible to log each key-value pair in the dictionary as a separate param to work around the current limitations of the UI?

[johny-c]

Hi @smurching, I think it makes more sence to log the params as a dictionary (a single json/yaml file). Flattening the dictionary would leave you with the question of a level separator that can only work for string keys. In any case, indexing/being able to search the config is what's important. Maybe now that databricks acquired redash, JSON makes even more sense, as I guess you can get some parsing, indexing, visualization, etc. for free.

[piclem]

Hi @johny-c , have you found a nice way of logging nested dictionaries parameters ? Have you thought of a UI element to "browse" such parameters (e.g. wrapping/unwrapping a tree of parameters...) ?

Since this is still not managed, I am thinking of using a dictionary "flattening" option (e.g. this SO question), that converts nested keys into flat "level1__level2" keys, that stay grouped together in the UI by alphabetical ordering.

param = {"level1": {"level2-key1": "value1", "level2-key2": "value2"}}
mlflow.log_params(flatten(param, separator='__'))

[rllyryan]

Any updates on this?

I am also facing this truncated view of a nested dictionaries when making run comparisons.

[selting]

neptune has a way of nesting logged parameters and metrics: https://docs.neptune.ai/logging/parameters/#nested-dictionary

It would indeed be nice to have such a feature in mlflow. A very natural purpose is to differentiate between training and test metrics. Currently, I think this requires a flattening approach as suggested by @piclem.

There are already dedicated 'Attributes' (Experiment Name, Source, ...), 'Parameters', and 'Metrics' in the UI. I'm not sure how they are treated under the hood, but it feels as if allowing subcategories in the user-populated 'Parameters' and 'Metrics' shouldn't be too hard.