Question on how to Instantiate a list of objects with multiple configs

[ryokan0123]

Hi, I find jsonargparse incredibly useful for organizing code in ML experiments. Thank you for creating such an excellent project.

I frequently encounter a use case where I need to pass a list of objects, each with separate config files. This would allow me to experiment with combinations of complex objects. However, I'm unsure if this is possible with the current version of jsonargparse.

For a minimal example, I'm looking to implement something like the following:

from dataclasses import dataclass
from typing import Any

import jsonargparse

@dataclass
class Foo:
    arg1: Any

@dataclass
class Bar:
    arg1: Any

@dataclass
class ComplexClass:
    foo: Foo
    bar: Bar


if __name__ == "__main__":
    parser = jsonargparse.ArgumentParser()
    parser.add_argument("--class_list", type=list[ComplexClass])

    args = parser.parse_args()
    args = parser.instantiate_classes(args)
    print(args)

And each ComplexClass would be configured through a separate config file like this:

foo:
  class_path: Foo
  init_args:
    arg: 1
bar:
  class_path: Bar
  init_args:
    arg: 2

Then the command would be:

python example.py --class_list+="complex_class.yaml"  --class_list+="complex_class2.yaml

Is there a way to achieve something like this? I understand that configuring through a single config file is possible, possibly by using jsonnet to import multiple config files, but being able to directly specify multiple config files through command-line arguments would be convenient.

[mauvilsa]

Currently that is not possible. Loading from subconfigs only works at argument level, not for each class in a list. A somewhat close alternative that would work now, although not ideal, is to use global config files with only the class you want to append to the list. The code would be:

from dataclasses import dataclass
from typing import Any

import jsonargparse

@dataclass
class Foo:
    arg1: Any

@dataclass
class Bar:
    arg1: Any

@dataclass
class ComplexClass:
    foo: Foo
    bar: Bar


if __name__ == "__main__":
    parser = jsonargparse.ArgumentParser()
    parser.add_argument("--config", action=jsonargparse.ActionConfigFile)
    parser.add_argument("--class_list", type=list[ComplexClass])

    args = parser.parse_args()
    args = parser.instantiate_classes(args)
    print(args)

be run as

python example.py --config=complex_class1.yaml --config=complex_class2.yaml

where each config would be like:

class_list+:
  - foo:
      arg1: 1
    bar:
      arg1: 2

Note that there are no class_path and init_args in the config. That is because dataclasses are not considered subclasses. This is until #287 is implemented and it becomes possible to select which class types should be considered or not subclasses.

[ryokan0123]

Loading from subconfigs is only effective at the argument level, not for individual classes within a list.

I see. This clarifies my question.

I will try the alternative solution you suggested. Thanks!