Pydantic Schema Validation Errors - on import

[Raziel90]

Hello!

I have been trying to use openpi as a package using the instructions provided.

Everytime iI try to import

openpi.policies.policy_config
openpi.training.config

I get the following problems:

openpi.policies.policy as _policy
Traceback (most recent call last):
  File ".../site-packages/pydantic/_internal/_generate_schema.py", line 2767, in finalize_schema
    gather_result = gather_schemas_for_cleaning(
  File ".../site-packages/pydantic/_internal/_schema_gather.py", line 199, in gather_schemas_for_cleaning
    traverse_schema(schema, context)
  ...
  File ".../site-packages/pydantic/_internal/_schema_gather.py", line 75, in traverse_definition_ref
    raise MissingDefinitionError(schema_ref)
pydantic._internal._schema_gather.MissingDefinitionError: any-shape-array-2b3b6d5522ac5a35

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../site-packages/external_lib/policies/policy.py", line 14, in <module>
    from external_lib import transforms as _transforms
  File ".../site-packages/external_lib/transforms.py", line 13, in <module>
    from external_lib.shared import normalize as _normalize
  File ".../site-packages/external_lib/shared/normalize.py", line 9, in <module>
    @pydantic.dataclasses.dataclass
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ...
  File ".../site-packages/numpydantic/schema.py", line 255, in make_json_schema
    list_schema = _handler._generate_schema.clean_schema(list_schema)
  File ".../site-packages/pydantic/_internal/_generate_schema.py", line 2772, in finalize_schema
    raise InvalidSchemaError from e
pydantic._internal._generate_schema.InvalidSchemaError 

what seems to be the problem:

numpydantic seems not to be able to validate the class in openpi.shared.normalize.py

@pydantic.dataclasses.dataclass
class NormStats:
    mean: numpydantic.NDArray
    std: numpydantic.NDArray
    q01: numpydantic.NDArray | None = None  # 1st quantile
    q99: numpydantic.NDArray | None = None  # 99th quantile

How I made it work

from typing import Annotated
@pydantic.dataclasses.dataclass(config=pydantic.ConfigDict(arbitrary_types_allowed=True))
class NormStats:
    mean: Annotated[np.ndarray, pydantic.Field()]
    std: Annotated[np.ndarray, pydantic.Field()]
    q01: Annotated[np.ndarray | None, pydantic.Field()] = None
    q99: Annotated[np.ndarray | None, pydantic.Field()] = None

[phbst]

I've encountered this problem, have you solved it?

[phbst]

pip install pydantic==2.10.6

i solved it.

[Raziel90]

@phbst Thanks for sharing your solution, mine was giving up in importing the package as an external package :D

[uzhilinsky]

Just to confirm, is upgrading from 2.10.4 to 2.10.6 is what solving this for you? (2.10.4 is what we currently install).

@Raziel90 , if you are still running into this, do you mind doing:

uv pip list | grep "pydantic"

It should print your current version.

[AasherH]

I had this same issue, upgraded pydantic to 2.10.6, and the problem was solved.