Comparing experiments with Hydra config shows wrong OmegaConf diff

[antifriz]

trafficstars

Instead of coloring only the lines that differ, every line after the first diff is colored too.

It seems like diff assumes the same line number too.

It really influences a lot the usability of Hydra for configuring experiments.

[jkhenning]

Hi @antifriz,

Thanks for reporting this, we'll take a look - can you provide a short example to replicate?

[erezalg]

Hi @antifriz, Please see below, I could not reproduce it.

Please let us know how to reproduce this, Thanks!

[aweinmann]

I have this issue too, for me it comes from experiments where an element is added to the config, like in the example above if you would add another field to the dataset in a new experiment it will color all rows below as changed even though they are all just offset by one line. Is there any workaround to this? Thanks

[ainoam]

@aweinmann Which clearml versions (SDK, server...) are you using? Can you share a small example to replicate?

[aweinmann]

Hi @ainoam I'm using the following version: Server: 1.12.1-397 SDK: 1.13.1

If i run the hydra example included and then, for example, add a seed field to dataset and run it again the compare will flag all lines below the inserted field as changed and it will look like this:

[aweinmann]

@ainoam any update on this issue?

[wxdrizzle]

+1 for this issue. I can reproduce it. It matters a lot if one like me frequently adds/deletes hyperparameters when designing new models.

My current solution is to manually convert the omegaconf object to a nested dictionary:

task.connect_configuration(OmegaConf.to_container(cfg))

Then the difference is shown correctly:

One drawback is that the nested dict format (with "{}" and "=") has a slightly lower readability, not as compact as the yaml format.