pydantic Argument 'bases' has incorrect type

Initial Checks

[X] I have searched GitHub for a duplicate issue and I'm sure this is something new
[X] I have searched Google & StackOverflow for a solution and couldn't find anything
[X] I have read and followed the docs and still think this is a bug
[X] I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description

So last night 1.10.13 got released. And apparently it breaks on runtime,

The context is that there are some integrations in HomeAssistant that use pydantic, and they all seem to raise this issue when 1.10.13 is installed. This isn't raised in 1.10.12 (https://github.com/home-assistant/core/pull/101044). Linked issue on the HA repo (https://github.com/home-assistant/core/issues/101042).

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 818, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/src/homeassistant/homeassistant/components/radio_browser/__init__.py", line 5, in <module>
    from radios import RadioBrowser, RadioBrowserError
  File "/usr/local/lib/python3.11/site-packages/radios/__init__.py", line 8, in <module>
    from .models import Country, Language, Station, Stats, Tag
  File "/usr/local/lib/python3.11/site-packages/radios/models.py", line 12, in <module>
    class Stats(BaseModel):
  File "pydantic/main.py", line 186, in pydantic.main.ModelMetaclass.__new__
TypeError: Argument 'bases' has incorrect type (expected list, got tuple)

Some of the libs used are https://github.com/frenck/python-vehicle and https://github.com/frenck/python-radios

Example Code

No response

Python, Pydantic & OS Version

Python 3.11
Pydantic 1.10.13
Linux

Affected Components

[ ] Compatibility between releases
[ ] Data validation/parsing
[ ] Data serialization - .model_dump() and .model_dump_json()
[ ] JSON Schema
[X] Dataclasses
[ ] Model Config
[ ] Field Types - adding or changing a particular data type
[ ] Function validation decorator
[ ] Generic Models
[ ] Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
[ ] Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

Sep 28 '23 11:09 joostlek

Do you have a minimal, reproducible example? We didn't change much: https://github.com/pydantic/pydantic/compare/v1.10.12...v1.10.13

Sep 28 '23 12:09 Kludex

I will give it a shot. As context, these are the libs in HA:

Warning!! Cyclic dependencies found:
* gps3 => gps3
------------------------------------------------------------------------
pydantic==1.10.13
├── aiolivisi==0.0.19 [requires: pydantic]
├── aionotion==2023.5.5 [requires: pydantic>=1.10.7,<2.0.0]
├── aioopenexchangerates==0.4.0 [requires: pydantic>=1.9,<2.0]
├── aiopurpleair==2022.12.1 [requires: pydantic>=1.10.2,<2.0.0]
├── aiowaqi==0.2.1 [requires: pydantic>=1.10.8]
├── demetriek==0.4.0 [requires: pydantic>=1.9.0,<2.0.0]
├── elgato==4.0.1 [requires: pydantic>=1.8.0,<2.0.0]
├── gcal-sync==4.1.4 [requires: pydantic>=1.9.0,<2.0a]
├── google-nest-sdm==3.0.2 [requires: pydantic>=1.10.4]
├── ical==5.0.1 [requires: pydantic>=1.9.1]
│   ├── gcal-sync==4.1.4 [requires: ical>=4.2.5]
│   └── pyrainbird==4.0.0 [requires: ical>=4.2.9]
├── inflect==7.0.0 [requires: pydantic>=1.9.1]
│   ├── jaraco.itertools==6.4.1 [requires: inflect]
│   │   └── jaraco.abode==3.3.0 [requires: jaraco.itertools]
│   └── jaraco.text==3.11.1 [requires: inflect]
│       ├── jaraco.collections==4.3.0 [requires: jaraco.text]
│       │   ├── jaraco.abode==3.3.0 [requires: jaraco.collections]
│       │   ├── jaraco.email==3.1.0 [requires: jaraco.collections]
│       │   │   └── jaraco.net==9.3.1 [requires: jaraco.email]
│       │   │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       │   └── jaraco.net==9.3.1 [requires: jaraco.collections]
│       │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       ├── jaraco.email==3.1.0 [requires: jaraco.text>=1.3]
│       │   └── jaraco.net==9.3.1 [requires: jaraco.email]
│       │       └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
│       └── jaraco.net==9.3.1 [requires: jaraco.text]
│           └── jaraco.abode==3.3.0 [requires: jaraco.net>=9]
├── intellifire4py==2.2.2 [requires: pydantic]
├── lacrosse-view==1.0.1 [requires: pydantic>=1.9.0]
├── open-meteo==0.2.1 [requires: pydantic>=1.8.0,<2.0.0]
├── peco==0.0.29 [requires: pydantic>=1.9.0]
├── pvo==1.0.0 [requires: pydantic>=1.8.0,<2.0.0]
├── pyaussiebb==0.0.15 [requires: pydantic>=1.9.0,<2.0.0]
├── pyrainbird==4.0.0 [requires: pydantic>=1.10.4]
├── python-bsblan==0.5.16 [requires: pydantic>=1.9.0]
├── python-kasa==0.5.3 [requires: pydantic>=1,<2]
├── python-opensky==0.2.0 [requires: pydantic>=1.10.8]
├── pytraccar==1.0.0 [requires: pydantic>=1,<2]
├── pyunifiprotect==4.20.0 [requires: pydantic!=1.9.1]
├── radios==0.1.1 [requires: pydantic>=1.9,<2.0]
├── sfrbox-api==0.0.6 [requires: pydantic>=1.10.2]
├── systembridgeconnector==3.8.2 [requires: pydantic>=1.9.0]
├── tailscale==0.2.0 [requires: pydantic>=1.8.0,<2.0.0]
├── vehicle==1.0.1 [requires: pydantic>=1.8.0,<2.0.0]
├── withings-api==2.4.0 [requires: pydantic>=1.7.2,<2.0.0]
├── xbox-webapi==2.0.11 [requires: pydantic]
├── yolink-api==0.3.1 [requires: pydantic>=1.9.0]
├── youtubeaio==1.1.5 [requires: pydantic>=1.10.8]
└── zwave-js-server-python==0.52.0 [requires: pydantic>=1.10.0]

We can already confirm from reports that these are failing:

zwave-js-server-python
pyunifiprotect
gcal-sync
ical
radios
vehicle
xbox-webapi
python-kasa
inflect
pytraccar
google-nest-sdm

I know this isn't an example, but all these libs raise problems when used with 1.10.13, and not with 1.10.12.

I'll get back with an example

Sep 28 '23 12:09 joostlek

We also looked at the changelog and were puzzled on why this suddenly fails

Sep 28 '23 12:09 joostlek

Apparently running tests of vehicle works on my windows machine with 1.10.13 installed. Also tried python-opensky.

Sep 28 '23 13:09 joostlek

Seems to only affect the systems we have binary wheels for ie i686/x86_64 previous report https://github.com/langchain-ai/langchain/issues/8361

Sep 28 '23 15:09 bdraco

pydantic-1.10.12 doesn't seem to have the problem, but I think they were built with cython 0.29.x

Maybe cython 3.x issue?

Sep 28 '23 16:09 bdraco

pydantic v1 should be built with cython 0.29:

https://github.com/pydantic/pydantic/blob/8822578619bf8d0bb754b1cf7a2a905b50240d01/requirements.txt#L3

Sep 28 '23 16:09 samuelcolvin

I think that won't get used until its in build-system

https://peps.python.org/pep-0518/#build-system-table

Sep 28 '23 16:09 bdraco

If I create a pyproject.toml

pydantic % cat pyproject.toml 
[build-system]
requires = ["setuptools", "wheel", "Cython==0.29.32;sys_platform!='win32'"]

It uses cython 0.29.32

Sep 28 '23 16:09 bdraco

Without the pyproject.toml its too late since setup.py loads cython and you get the system installed version instead of the one to build with

Sep 28 '23 16:09 bdraco

makes sense. @hramezani could you work on another patch release using the right cython when building.

Sep 28 '23 16:09 samuelcolvin

https://github.com/pydantic/pydantic/pull/7695

Edit: saw above comment after I opened the PR. Closed the pr

Sep 28 '23 16:09 bdraco

Hi, I've tried to reproduce this using and x64_64 instruction set using gitpod (I'm on an M1 mac today), and I can't reproduce it.

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.10.13
            pydantic compiled: True
                 install path: /workspace/.pyenv_mirror/user/current/lib/python3.11/site-packages/pydantic
               python version: 3.11.5 (main, Sep 15 2023, 13:46:24) [GCC 11.4.0]
                     platform: Linux-6.1.54-060154-generic-x86_64-with-glibc2.35
     optional deps. installed: ['typing-extensions']

I just tried creating a model and doing some validation and it ran fine - but the exception occurs on this line which is just creating a model.

I'd really like to be able to reproduce this, then confirm it's fixed before creating a new release.

Sep 29 '23 13:09 samuelcolvin

I also tried running pydantic 1.10.13 inside alpine inside gitpod to use musl binaries, and again - no error.

Sep 29 '23 13:09 samuelcolvin

same here, tried do install dietpi's octoprint on a pi 2 zero

Screen outputTraceback (most recent call last): File "/mnt/dietpi_userdata/octoprint/.local/bin/octoprint", line 8, in <module> sys.exit(main()) ^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/__init__.py", line 1008, in main octo(args=args, prog_name="octoprint", auto_envvar_prefix="OCTOPRINT") File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/cli/common.py", line 37, in invoke return self._impl.invoke(ctx) ^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1685, in invoke super().invoke(ctx) File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/cli/config.py", line 57, in cli ctx.obj.settings = init_settings( ^^^^^^^^^^^^^^ File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/__init__.py", line 203, in init_settings from octoprint.settings import InvalidSettings, settings File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/settings/__init__.py", line 35, in <module> from octoprint.schema.config import Config File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/schema/config/__init__.py", line 9, in <module> from .access_control import AccessControlConfig File "/mnt/dietpi_userdata/octoprint/.local/lib/python3.11/site-packages/octoprint/schema/config/access_control.py", line 11, in <module> class AccessControlConfig(BaseModel): File "pydantic/main.py", line 186, in pydantic.main.ModelMetaclass.__new__ TypeError: Argument 'bases' has incorrect type (expected list, got tuple) root@DietPi:/tmp/DietPi-Software# sudo -u octoprint /mnt/dietpi_userdata/octoprint/.local/bin/octoprint config set server.port 5001^C

Oct 03 '23 08:10 rkoehler2017

In case it helps since (a) I spotted this issue reading the OctoPrint release notes, and (b) I just spent a some time debugging a mysterious issue with importlib and TypeError that was hard to reproduce in another Python project (https://github.com/GlasgowEmbedded/glasgow/issues/422): early Python 3.9 / especially early Python 3.10 don't have some fixes to importlib that got backported, eg, around March 2022 (eg, https://github.com/python/cpython/issues/91160).

In the case of the problem I was debugging (metadata extras) the problem was a re.Match() was being returned, instead of a string, it affected Python 3.9 <= 3.9.10 and Python 3.10 <= 3.10.2. Your issue is clearly slightly different (tuple instead of list, and bases) but I'd suggest considering "maybe this issue got fixed in a Python importlib backport", and looking for a potential backport fix (I found the one affecting me via git annotate on the importlib function being called).

In the Glasgow project case I initially worked around it by catching the return type received was the "wrong one" and fixing it up before using it in the Glasgow project code; but the final fix merged ended up monkeypatching in the newer code for importlib metadata extras only on the old affected Python minor versions (see https://github.com/GlasgowEmbedded/glasgow/pull/436). (in the tuple / list case, you could fairly easily unconditionally force it to be a list before using it.)

Ewen

Oct 09 '23 18:10 ewenmcneill

It gets weirder.

After pinning pydantic to 1.10.12 in my project (OctoPrint) to solve this problem, and confirming it against various deployment scenarios, I got reports that even that won't work on RaspberryPi OS Bookworm with Python 3.11.2.

So I ran some tests:

Python 3.11.4 on x86_64 (my laptop): works
CI (GitHub Actions) with Python 3.11.2 on x86_64: works
RPiOS Buster with Python 3.9 on armv7: works
armv7 Docker with stock Debian Bookworm packages and Python 3.11.2: works
armv7 Docker with RPiOS Bookworm packages and Python 3.11.2: broken

I had to downgrade to pydantic 1.10.10 in the last scenario to make things work there again.

At this point I have to say that I'm beyond confused by this particular issue but hope that the above findings can help in any way to get to the bottom of this.

Oct 12 '23 09:10 foosel

We have also seen reports of 1.10.12 not working for some reason. In our case that could've been caused by custom integrations (which have their own dependency), but in earlier tests, installing 1.10.12 would fix the issue, but in some cases it just doesn't

Oct 12 '23 09:10 joostlek

Well, if anything at least it fits spooky season 🎃👻

Currently running some more tests to collect some more data points.

Oct 12 '23 09:10 foosel

I got some more results. On a whim I tried a stock Debian Bookworm image with an added config to use piwheels.org as additional index for pip, and THAT resulted in a reproduction.

My test setup can be found here: https://gist.github.com/foosel/80b22908c008c063b85c819aee12c5ad

I'm testing with a really minimal test.py:

from pydantic import BaseModel


class TestModel(BaseModel):
    foo: str = "bar"


if __name__ == "__main__":
    test = TestModel()
    print(test)

And here's the results against Debian Bookworm with Python 3.11.2 on arm32v7:

pydantic	pypi	piwheels
1.10.10	✅	✅
1.10.11	✅	❌
1.10.12	✅	❌
1.10.13	✅	❌

So the issue as observed for pydantic 1.10.11 through 1.10.13 on Raspberry Pi OS Bookworm with Python 3.11.2 seems to stem from the binary wheels provided by the piwheels project. Piwheels comes preconfigured as additional index URL on RPi OS, and telling pip to use piwheels als additional index via /etc/pip.conf results on failure for pydantic 1.10.11 through 1.10.13 even on a stock Debian Bookworm under arm32v7.

A look at the package page reveals that piwheels only has wheels available from 1.10.11 onward, which means 1.10.10 would be taken from pypi, so something is apparently up with the builds on piwheels against Python 3.11.

I wonder if the packages compiled by piwheels also ran into the aforementioned problem of accidentally being compiled against Cython 3?

edit In any case, someone from the pydantic team with the proper authority of a maintainer should maybe request removal of the affected packages from piwheels as described in the FAQ:

edit I took the liberty to at least report the problem to piwheels: piwheels/packages#396

Oct 12 '23 10:10 foosel

I also tried running pydantic 1.10.13 inside alpine inside gitpod to use musl binaries, and again - no error.

Updating from 1.10.12 to 1.10.13 fixed the problem for me.

Nov 04 '23 20:11 CaseyHaralson

I see this issue while testing maison.

Nov 09 '23 16:11 mtelka

Testing of jaraco.text 3.12.0 with pydantic 1.10.13 failed:

________________ ERROR collecting jaraco/text/show-newlines.py _________________
/usr/lib/python3.9/vendor-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
/usr/lib/python3.9/vendor-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/usr/lib/python3.9/vendor-packages/_pytest/doctest.py:567: in collect
    module = import_path(
/usr/lib/python3.9/vendor-packages/_pytest/pathlib.py:567: in import_path
    importlib.import_module(module_name)
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1030: in _gcd_import
    ???
<frozen importlib._bootstrap>:1007: in _find_and_load
    ???
<frozen importlib._bootstrap>:986: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:680: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:850: in exec_module
    ???
<frozen importlib._bootstrap>:228: in _call_with_frames_removed
    ???
jaraco/text/show-newlines.py:2: in <module>
    import inflect
/usr/lib/python3.9/vendor-packages/inflect/__init__.py:2027: in <module>
    class engine:
/usr/lib/python3.9/vendor-packages/inflect/__init__.py:2049: in engine
    def defnoun(self, singular: Optional[Word], plural: Optional[Word]) -> int:
pydantic/decorator.py:49: in pydantic.decorator.validate_arguments
    ???
pydantic/decorator.py:36: in pydantic.decorator.validate_arguments.validate
    ???
pydantic/decorator.py:126: in pydantic.decorator.ValidatedFunction.__init__
    ???
pydantic/decorator.py:264: in pydantic.decorator.ValidatedFunction.create_model
    ???
pydantic/main.py:1026: in pydantic.main.create_model
    ???
pydantic/main.py:186: in pydantic.main.ModelMetaclass.__new__
    ???
E   TypeError: Argument 'bases' has incorrect type (expected list, got tuple)

Nov 28 '23 10:11 mtelka

While the smart people are working on the proper fix for this, is there a simple workaround that we can use?

Feb 18 '24 00:02 johntamplin

pydantic pydantic copied to clipboard

Argument 'bases' has incorrect type

Initial Checks

Description

Example Code

Python, Pydantic & OS Version

Affected Components

pydantic
pydantic copied to clipboard