omegaconf icon indicating copy to clipboard operation
omegaconf copied to clipboard

Allow for automatic `.` separated key conversion to nested dictionary

Open puneeter opened this issue 1 year ago • 5 comments

Describe the bug

In[25]: oc
Out[25]: {'a.b': 1, 'a.c': 2}

In[26]: OmegaConf.select(oc, "a.b")

In[27]: OmegaConf.select(oc, ".a.b")

In[28]: OmegaConf.select(oc, ".")
Out[28]: {'a.b': 1, 'a.c': 2}

There should be a way to specify . separated namespaces in my config and OmegaConf should ideally split all these keys into nested dictionaries.

To Reproduce See in the above section

Expected behavior

For the above code I expect the following output:

In[26]: OmegaConf.select(oc, "a.b")
Out[26]: {'a.b': 1}

In[27]: OmegaConf.select(oc, ".a.b")
Out[27]: {'a.b': 1}

Additional context

  • [x] OmegaConf version: 2.3.0
  • [x] Python version: 3.9.19
  • [x] Operating system: Mac OS
  • [x] Please provide a minimal repro

puneeter avatar Aug 14 '24 10:08 puneeter

related: #1189

noklam avatar Aug 14 '24 11:08 noklam

There should be a way to specify . separated namespaces in my config and OmegaConf should ideally split all these keys into nested dictionaries.

Currently it is allowed (but not recommended) to use dots in config keys. This makes OmegaConf compatible with a wider range of use cases, and I don't foresee it changing in the future.

That being said, it may be useful to allow creating a config from a dict whose keys contains dot and automatically turning it into a nested config. This couldn't just use the syntax from #1189 as it would break backward compatibility, but this could be either a new parameter to create(), or a new function.

In the meantime, my suggestion is that you implement this function yourself -- you could then share it here so that someone else looking for the same feature can re-use it directly :)

odelalleau avatar Aug 14 '24 14:08 odelalleau

Thanks for responding @odelalleau ! Would you be open to have this functionality embedded in the create method?

puneeter avatar Aug 14 '24 14:08 puneeter

Thanks for responding @odelalleau ! Would you be open to have this functionality embedded in the create method?

Potentially yes, but only if some additional flag is needed to enable this behavior, since we shouldn't break the existing default behavior which is working as intended.

(EDIT: as a result I may lean more towards a different dedicated function, like OmegaConf.create_from_dot_keys() -- since I don't think that a special flag would make sense for the non-dict inputs that OmegaConf.create() currently accepts)

odelalleau avatar Aug 14 '24 17:08 odelalleau

OmegaConf.select treats "." as a separator. In theory it could be possible to extend it to support escaping of the . (e.g. select(cfg, "a\.b") ), but this is not something that is likely to be included.

omry avatar Aug 25 '24 14:08 omry