Support interpolation to integer keys (or other non-string keys)

[NatanBagrov]

I want to achieve strong coupling between several configuration components. For example, in Image Detection, each dataset/resolution combination has best fitting "anchors". I have a config with dataset.resolution.anchors with anchors. Now, I want to couple this key with the specific dataset and resolution in the "outer" config.

# demo.py

from omegaconf import OmegaConf

yaml_data = """
anchors:
  coco2014:
    224:
        data: [1,2,3]
    512:
        data: [2,3,4]
  coco2017:
    224:
        data: [3,4,5]
    512:
        data: [4,5,6]

dataset: coco2014
resolution: 224
ref: ${anchors.${dataset}.${resolution}.data}  # THIS IS WHAT I WANT TO ACHIEVE
"""

cfg = OmegaConf.create(yaml_data)
print(cfg.ref)  # raises InterpolationResolutionError

Describe the solution you'd like A nested interpolation: ${anchors.${dataset}.${resolution}.data}

Describe alternatives you've considered Normal interpolation: $anchord.coco2014.224.data (note that integers are not working as well, so I workaround with 224x224 instead of 224 in the relevant parts) But that is bad, because if you change resolution: 512 then you basically must change the interpolation as-well -- a recipe for bugs.

[Jasha10]

As you mentioned, OmegaConf interpolations do not handle integers well. The root cause here is that the interpolation machinery uses strings to represent the address to interpolate to, so typed values like int or float are not supported.

One possible workaround is to ensure that all the relevant data are strings (as opposed to integers) by using quotation marks:

# demo2.py
from omegaconf import OmegaConf

yaml_data = """
anchors:
  coco2014:
    "224":
        data: [1,2,3]
    "512":
        data: [2,3,4]
  coco2017:
    "224":
        data: [3,4,5]
    "512":
        data: [4,5,6]

dataset: coco2014
resolution: "224"
ref: ${anchors.${dataset}.${resolution}.data}
"""

cfg = OmegaConf.create(yaml_data)
print(cfg.ref)  # prints [1, 2, 3]

But that is bad, because if you change resolution: 512 then you basically must change the interpolation as-well -- a recipe for bugs.

I'm not sure exactly what you mean here. Does my suggestion above circumvent the potential bugs that you foresee?

[NatanBagrov]

So it appears like the root cause is indeed integers. The suggestion is ok, but then the problem goes to the resolution (that the pipeline assumes it is an integer). I guess the new issue name "Support interpolation to integer keys (or other non-string keys)" is exact. Will wait. Thanks!

[pixelb]

Well at least the error is quite clear as to the issue:

raise InterpolationResolutionError(
omegaconf.errors.InterpolationResolutionError:
The following interpolation is used to denote a config key and thus should return a string,
but instead returned `224` of type `<class 'int'>`: ${resolution}

foo.bar (__getattr__) is implicitly strings only in python, so we would need to carefully consider the generality of changes here

[NatanBagrov]

One workaround (maybe even the proper way) is to use getattr explicitly and wrap the parameter (attribute name) with str(...)?

[Jasha10]

foo.bar (getattr) is implicitly strings only in python, so we would need to carefully consider the generality of changes here

Good point.

One workaround (maybe even the proper way) is to use getattr explicitly and wrap the parameter (attribute name) with str(...)?

I am not fond of this idea because converting the key type could lead to ambiguity in cases where string keys are mixed with integer keys, e.g. dicts such as {"123": "string data", 123: "integer data"}.

Note that a __getitem__-like bracket syntax is already supported by OmegaConf's interpolation grammar:

cfg = OmegaConf.create({
    "foo": {"bar": "baz"},
    "qux": "${foo[bar]}",
})
assert cfg.qux == 'baz'

This means that the grammar could support __getitem__ syntax for non-string keys. Here is a possible API:

data:
   "123": "string data"
   123: "integer data"

a_string: "123"
an_int: 123

ref1: ${data.${a_string}}   # currently working
ref2: ${data.${an_int}}     # raises InterpolationResolutionError (which I think is ok as-is)
ref3: ${data[${a_string}]}  # currently working
ref4: ${data[${an_int}]}    # not working currently. Proposed!

[NatanBagrov]

I agree regarding the ambiguity. Maybe square brackets should be the go-to if we're using integers.