Data repetition in get_multi_joined and issue in join_on

[mithun2003]

data = crud_user.get_multi(
    db,
    nest_joins=True,
    joins_config=[
        JoinConfig(
            model=Portions,
            join_on=User.portion_id == Portion.id,
            join_prefix='portions',
            join_type="left",
           relationship_type='one-to-many'
        ),
        JoinConfig(
            model=Category,
            join_on=User.created_id == Category.id,
        ),
    ],
)

the above code is what I used to run. When it is complete, it give data like this but the problem is that in portions, the data is repeating like the below one;

{
    "id": 1,
    "created_user_id": 1,
    "name": "Test",
    "email": "[email protected]",
    "portions": [
        {"id": 1, "user_id": 1, "quantity": "Half"},
        {"id": 2, "user_id": 1, "quantity": "Full"},
        {"id": 1, "user_id": 1, "quantity": "Half"},
        {"id": 2, "user_id": 1, "quantity": "Full"},
    ],
    "category": [{"id": 1, "category": "Food"}, {"id": 2, "category": "Real-Estate"}],
}

when i debug the code, the problem is found in here;

def _nest_multi_join_data(
    base_primary_key: str,
    data: list[Union[dict, BaseModel]],
    joins_config: Sequence[JoinConfig],
    return_as_model: bool = False,
    schema_to_select: Optional[type[BaseModel]] = None,
    nested_schema_to_select: Optional[dict[str, type[BaseModel]]] = None,
) -> Sequence[Union[dict, BaseModel]]:

    pre_nested_data = {}

    for join_config in joins_config:
        join_primary_key = _get_primary_key(join_config.model)

        for row in data:
            if isinstance(row, BaseModel):
                new_row = {key: (value[:] if isinstance(value, list) else value) for key, value in row.model_dump().items()}
            else:
                new_row = {key: (value[:] if isinstance(value, list) else value) for key, value in row.items()}

            primary_key_value = new_row[base_primary_key]

            if primary_key_value not in pre_nested_data:
                for key, value in new_row.items():
                    if isinstance(value, list) and any(item[join_primary_key] is None for item in value):  # pragma: no cover
                        new_row[key] = []

                pre_nested_data[primary_key_value] = new_row
            else:
                existing_row = pre_nested_data[primary_key_value]
                for key, value in new_row.items():
                    if isinstance(value, list):
                        if any(item[join_primary_key] is None for item in value):  # pragma: no cover
                            existing_row[key] = []
                        else:
                            existing_row[key].extend(value)

    nested_data: list = list(pre_nested_data.values())

here the

for join_config in joins_config:
        join_primary_key = _get_primary_key(join_config.model)

        for row in data:

When the first loop is complete, it gives the correct data, but the problem is that there are two join configs so the outside loop will execute again and do the same thing. I think that's why the data is repeating

---

And also, there is another issue, like if I add join_on, it will show this error message: Could not automatically determine join condition. Please provide join_on. The reason is because: join_on=join_on or _auto_detect_join_condition(self.model, join_model), To solve it, I added it like this:

if join_on is None and joins_config is None:
            join_on = _auto_detect_join_condition(self.model, join_model)

[igorbenav]

Hey, @mithun2003, thanks for the issue! Do you want to fix it as well?

[mithun2003]

Hey @igorbenav, Well I have been busy for a while now. But I will check on this issue when I got some time.

[igorbenav]

Closed by #186