feat: Add Support for complex list defined from env vars

[inean]

Currently , only list of complex types can only be defined completely from env vars as Json values. The idea is to also allow to define using nested delimiters, where index is explicitly defined in env vars:

class SubType(BaseSettings):
    v1 :str 

class Config(Settings):
   top: list[SubType]

then, something like this should work...

CONFIG__0__TOP="foo"

[inean]

PR #377

[MarcBresson]

linked issue

[plpilew]

As a workaround for this problem I used dict with integer key instead of a list:

class SubType(BaseSettings):
    v1 :str 

class Config(Settings):
   top: dict[int, SubType]

Then I can use such environment variable to override the value:

CONFIG__TOP__0__V1="foo"

Still not perfect, but at least our Kubernetes developers can use same patterns for naming env variables regardless whether the pod contains python, dotnet or other stack (using index to override array items works in dotnet).

[MarcBresson]

@plpilew please see my comment that has the benefit of being a list.

[plpilew]

@plpilew please see my comment that has the benefit of being a list.

I tried this approach but could not use it for my scenario which is:

1. My model (SubType in this case) has multiple fields.
2. Configuration is read from .env file which sets all the fields.
3. Then I want one of the fields to be updated to the value read from env variable leaving other fields intact.

In this case the built-in pydantic validation raised an error saying that other fields are required as well (I had only one env variable defined).

[MarcBresson]

What about :

from typing import TypeVar

from pydantic import BaseModel


T = TypeVar("T")


def env_var_dict_to_list(model: BaseModel, data: dict[str, T] | list[T]) -> list[T]:
    """If the data is a dictionary, convert its values to a list.

    It allows for the definition of lists through env var in pydantic
    settings.

    For instance, given the following BaseModel:
    ```python
    >>> from pydantic import BaseModel

    >>> class Model(BaseModel):
    >>>     slug: str
    >>>     weight: float
    ```

    You could define the following environment variables:

    ```
    models__0__slug=my-super-model
    models__0__weight=0.1
    models__1__slug=a-second-impressive-model
    models__1__weight=0.4
    ```

    This function will convert the dictionary given by pydantic-settings to a
    list of Model instances:
    ```python
    >>> from typing import Annotated
    >>> from pydantic import BeforeValidator
    >>> from pydantic_settings import BaseSettings
    >>> from your_module import env_var_dict_to_list
    >>> from your_model import Model

    >>> class Settings(BaseSettings):
    >>>     models: Annotated[
    >>>         list[Model],
    >>>         BeforeValidator(lambda v: env_var_dict_to_list(Model, v)),
    >>>     ]

    >>> Settings()
    Settings(
        models=[
            Model(slug='my-super-model', weight=0.1),
            Model(slug='a-second-impressive-model', weight=0.4)
        ]
    )
    ```
    ```
    """
    if isinstance(data, dict):
        return [model(**item) for item in data.values()]
    return data

and then, in your model:

from pydantic import BaseModel, BeforeValidator


class SubType(BaseModel):
    v1: str


class Config(BaseSettings):
    model_config = ConfigDict(env_nested_delimiter="__")

    sub_types: Annotated[list[Model, ...], BeforeValidator(lambda v: env_var_dict_to_list(SubType, v))]

I must admit that your solution is much (much) simpler than this one.