omegaconf icon indicating copy to clipboard operation
omegaconf copied to clipboard

Subclasses of `Generic` cannot be used as structured configs

Open TidalPaladin opened this issue 4 years ago • 6 comments

Describe the bug Creating a structured config from a subclass of typing.Generic fails with IndexError: tuple index out of range. Having a member variable annotated as TypeVar("T") is not required to produce the error.

To Reproduce

from typing import Generic, TypeVar
from dataclasses import dataclass
from omegaconf import OmegaConf

T = TypeVar("T")

@dataclass
class Base(Generic[T]):
    x: int = 1 
    # error when x: T as well

@dataclass
class Derived(Base[int]):
    ...
    # also an error when Base.x is annotated as T and Derived.x is annotated as int

Conf = OmegaConf.structured(Derived)

Fails with

Traceback (most recent call last):
  File "bug.py", line 17, in <module>
    Conf = OmegaConf.structured(Derived)
  File "/home/tidal/Documents/combustion/venv/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 119, in structured
    return OmegaConf.create(obj, parent, flags)
  File "/home/tidal/Documents/combustion/venv/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 172, in create
    return OmegaConf._create_impl(
  File "/home/tidal/Documents/combustion/venv/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 825, in _create_impl
    key_type, element_type = get_dict_key_value_types(obj_type)
  File "/home/tidal/Documents/combustion/venv/lib/python3.8/site-packages/omegaconf/_utils.py", line 640, in get_dict_key_value_types
    element_type = args[1]
IndexError: tuple index out of range

Expected behavior Either structured config creation should work with subclasses of Generic, or the raised exception should indicate that this is unsupported.

A note on use case: I ran into this while playing around with type annotations for Hydra. I wanted to see if it was possible to create structured configs with type annotations that describe the output of instantiate. Something like

@dataclass
class MyModelConf(Instantiable[MyModel]):
    _target_: str = "module.MyModel"
    in_channels: int = 3
    ...
    optimizer: Instantiable[torch.optim.Optimizer] = AdamConf()

Beyond this use case, I'm not sure what demand there is for creating structured configs from subclasses of Generic.

Additional context

  • [ ] OmegaConf version: 2.1.0.rc1
  • [ ] Python version: 3.8.5
  • [ ] Operating system: Linux
  • [ ] Please provide a minimal repro

TidalPaladin avatar May 24 '21 20:05 TidalPaladin

Thanks for the report!

Jasha10 avatar May 26 '21 19:05 Jasha10

This error comes from the mechanism that OmegaConf uses to inspect base classes of Structured Config classes; the inspection occurs so that structured configs that are subclasses of typing.Dict can recieve special treatment.

Since support for subclassing typing.Dict is deprecated and is planned for removal in OmegaConf 2.2, I expect this issue to solve itself once we reach 2.2.

Jasha10 avatar May 26 '21 20:05 Jasha10

Dependent on issue #821

pixelb avatar Apr 07 '22 17:04 pixelb

The issue is still happening on version 2.3.0

lior1990 avatar Jan 17 '23 09:01 lior1990

Yes indeed, we have not pulled the trigger on removing the deprecated special treatment for subclasses of typing.Dict.

Jasha10 avatar Jan 17 '23 12:01 Jasha10