Question about parse a list of dict

[bot66]

I'm trying to parse a arg that is a list of dict use the comand line tool:

The yaml:

# part of the config.yaml
model:
    augmentations:
    - a: random_crop
    - b: random_blur

Expected result:

# part of the config.yaml
model:
    augmentations:
    - c: random_mosaic
    - d: random_perspective

The command line arg I try: 1.

--model.augmentations+={"c":"random_mosaic"} --model.augmentations+={"d":"random_perspective"}

#Error message
    Parser key "augmentations":
      Expected a <class 'dict'>. Got value: {c:random_mosaic}

--model.augmentations=[{"c":"random_mosaic"},{"d":"random_perspective"}]

#Error message
    Parser key "augmentations":
      Expected a <class 'str'>. Got value: None

How to properly parse a list of dict use cli?

[mauvilsa]

This is because your shell is modifying the values before they get to the python code. You need to quote them to prevent this.

$ ./issue_421.py --model.augmentations+='{"c":"random_mosaic"}' --model.augmentations+='{"d":"random_perspective"}'
Namespace(model=Namespace(augmentations=[{'c': 'random_mosaic'}, {'d': 'random_perspective'}]))

$ ./issue_421.py --model.augmentations='[{"c":"random_mosaic"},{"d":"random_perspective"}]'
Namespace(model=Namespace(augmentations=[{'c': 'random_mosaic'}, {'d': 'random_perspective'}]))

[bot66]

This is because your shell is modifying the values before they get to the python code. You need to quote them to prevent this.

$ ./issue_421.py --model.augmentations+='{"c":"random_mosaic"}' --model.augmentations+='{"d":"random_perspective"}'
Namespace(model=Namespace(augmentations=[{'c': 'random_mosaic'}, {'d': 'random_perspective'}]))

$ ./issue_421.py --model.augmentations='[{"c":"random_mosaic"},{"d":"random_perspective"}]'
Namespace(model=Namespace(augmentations=[{'c': 'random_mosaic'}, {'d': 'random_perspective'}]))

Thank you. The second one works. But the first one did not work.

--model.augmentations+='{"c":"random_mosaic"}' --model.augmentations+='{"d":"random_perspective"}'

#Error message
error: 'Configuration check failed :: No action for destination key "augmentations+.c" to check its value.'

[mauvilsa]

I can look into what happened with the first one. But you need to provide the exact type hint of augmentations and say which shell you are using. Maybe also which operating system.

[bot66]

I can look into what happened with the first one. But you need to provide the exact type hint of augmentations and say which shell you are using.

The type hint:

    def __init__(
        self,
        augmentations: list[dict[str,str]] = [{"a":"random_crop"},{"b":"random_blur"}],
    ) -> None:

The shell I am using is /bin/bash on Ubuntu 22.04.2 LTS And the jsonargparse version is 4.21.0, I use it with LightningCLI .

[mauvilsa]

Why do you use an old version of jsonargparse? Not sure if something related to this has been fixed since 4.21, but better if you install the latest version.

I haven't had time to look in more detail. But key "augmentations+.c" looks like a bug in jsonargparse. In your LightningCLI your model is in subclass mode or not?

[bot66]

Hi, @mauvilsa

The LightningCLI subclass_mode_model and subclass_mode_data is False

And I upgraded jsonargparse to 4.27.0, still have errors:

error: Validation failed: No action for key "augmentations+.c" to check its value.

[mauvilsa]

With

from lightning.pytorch.demos.boring_classes import BoringModel
from lightning.pytorch.cli import LightningCLI

class MyModel(BoringModel):
    def __init__(
        self,
        augmentations: list[dict[str,str]] = [{"a":"random_crop"},{"b":"random_blur"}],
    ) -> None:
        pass

if __name__ == '__main__':
    cli = LightningCLI(MyModel, subclass_mode_model=False)

In ubuntu 22.04 and python 3.10 I get the following:

$ ./issue_421.py fit --model.augmentations+='{"c":"random_mosaic"}' --model.augmentations+='{"d":"random_perspective"}' --print_config
...
model:
  augmentations:
  - a: random_crop
  - b: random_blur
  - c: random_mosaic
  - d: random_perspective
...

However, if subclass_mode_model=True then I get the following

$ ./issue_421.py fit --model=MyModel --model.init_args.augmentations+='{"c":"random_mosaic"}' --model.init_args.augmentations+='{"d":"random_perspective"}' --print_config
usage: issue_421.py [-h] [-c CONFIG] [--print_config[=flags]] {fit,validate,test,predict} ...
error: Validation failed: No action for key "augmentations+.c" to check its value.

Did you mean to say that subclass_mode_model is True?

[bot66]

Hi, @mauvilsa

Appears the subclass_mode_model is True, although I did not explicitly set it to True, since I did not provide the model_class, it is set to True in the __init__()

self.subclass_mode_model = (model_class is None) or subclass_mode_model