Translate param.Parameterized to pydantic.BaseModel

[ahuang11]

Jim mentioned adding what Marc and I shared on Discourse to param, perhaps as a util(?), or simply briefly mention in docs. https://discourse.holoviz.org/t/how-would-i-create-a-parameterized-class-programmatically/3149/4?u=ahuang11

Note, this does not handle converting param methods at all; simply incompletely translating the parameters into fields.

The opposite issue (converting from pydantic -> param) can be found here: https://github.com/holoviz/param/issues/809

[ahuang11]

I think we should migrate https://github.com/holoviz/lumen/blob/main/lumen/ai/translate.py to param someday to have a method on Parameterized.to_pydantic_base_model() or something.

from pydantic import BaseModel, Field
from typing import Union


class Pet(BaseModel):
    name: str = Field(..., description="Name of the pet")
    age: int = Field(..., ge=0, le=20, description="Age of the pet in years")


class Dog(Pet):
    bark_volume: int = Field(default=5, ge=0, le=10)


class Cat(Pet):
    meow_pitch: float = Field(default=0.5, gt=0, lt=1)


class Owner(BaseModel):
    pet: Union[Dog, Cat]


Owner.schema()

import param
from lumen.ai.translate import param_to_pydantic


class Pet(param.Parameterized):
    name = param.String(doc="Name of the pet")
    age = param.Integer(default=1, bounds=(0, 20), doc="Age of the pet in years")

class Dog(Pet):
    bark_volume = param.Integer(default=5, bounds=(0, 10))


class Cat(Pet):
    meow_pitch = param.Number(default=0.5, bounds=(0, 1))


class Owner(param.Parameterized):
    pet = param.ClassSelector(class_=(Dog, Cat))


response_model = param_to_pydantic(Owner, excluded=["name"])["Owner"]
response_model.schema()

[MarcSkovMadsen]

There is also this related PR https://github.com/holoviz/panel/pull/6912 .

[ahuang11]

Should probably bump that PR heh

[MarcSkovMadsen]

The PR is amazing. Also makes interacting with ipywidgets much, much better. Cant wait.

[maximlt]

I think we should migrate https://github.com/holoviz/lumen/blob/main/lumen/ai/translate.py to param someday to have a method on Parameterized.to_pydantic_base_model() or something.

Just to make sure, is this just about pydantic_to_param? As the module contains more utilities.

Then I'd say:

• Pydantic would be an optional dependency of param, available via an extra (pip install param[pydantic])
• It will need to be well tested (there aren't many tests in https://github.com/holoviz/lumen/blob/main/lumen/tests/ai/test_translate.py)
• There should be a minimum pin for the version of pydantic we support (e.g. pydantic >= 2.10.0). The tests should run with that version, and with the latest version of pydantic, to ensure we maintain compatibility
• Need to find a place where to document it. The docstring of pydantic_to_param says Serialize a Pydantic model to a Parameterized instance but I don't think that's the usual definition of serializing and so am unsure it would be a good fit for the serialization guide.

Quick question on the implementation, does the converter just focus on converting fields to parameters (ignoring methods, class variables, etc.)?

Do we expect other "converters" like this to be added to Param (dataclass, traitlets, etc.)? I can imagine a converter.py module similar to serializer.py.

[Coderambling]

The PR is amazing. Also makes interacting with ipywidgets much, much better. Cant wait.

Do you mean that PR might actually be merged soon(ish)? Is it basically complete, but dependent on another change?

[ahuang11]

It will need to be well tested (there aren't many tests in https://github.com/holoviz/lumen/blob/main/lumen/tests/ai/test_translate.py)

I agree.

Quick question on the implementation, does the converter just focus on converting fields to parameters (ignoring methods, class variables, etc.)?

Early stage yes, but maybe in the future, we can consider migrating methods too.

[maximlt]

Early stage yes, but maybe in the future, we can consider migrating methods too.

Ok. For forward compatibility, might be good to add a kwarg like ignore_methods=True in the first implementation (warning/error if set to False).

[MarcSkovMadsen]

The PR is amazing. Also makes interacting with ipywidgets much, much better. Cant wait.

Do you mean that PR might actually be merged soon(ish)? Is it basically complete, but dependent on another change?

To me it was complete. I think the Challenge is more to figure out if this is core Panel or an extension because its a comprehensive PR.

[jbednar]

Do we expect other "converters" like this to be added to Param (dataclass, traitlets, etc.)?

dataclass support makes great sense and I'd be strongly in favor. Traitlets also makes sense but would depend on identifying some specific clear use cases.

[jbednar]

I think the Challenge is more to figure out if this is core Panel or an extension because its a comprehensive PR.

Core Param, you mean? I'm ok with having specific separate .py files that are ambitious like this included with the main library, as long as they are optional and don't add any dependencies unless they are used.

[maximlt]

They're chatting about this panel PR https://github.com/holoviz/panel/pull/6912

[jbednar]

That's confusing!

[Coderambling]

I also find this confusing. I would be great if someone could clarify if / how do the PR's relate to each other?

So:

PR #828 achieves the below, as stated near the start of the comments?

simply incompletely translating the parameters into fields.

Or ... ?

The other PR https://github.com/holoviz/panel/pull/6912 achieves this:

...

So they their relation is ... (or maybe they are not really, or only superficially related?

Great work by the way in both PR's @MarcSkovMadsen ! It would unleash so much potential if this PR would be realized.

[Coderambling]

I also feel this PR would increases the chances of param being adopted outside of Holoviz. Is that the case?

Is it indeed important for the param library to have (more) adoption outside of Holoviz?

IDE support for param would be another important element in that context.