replicate
Fix Pydantic v1 incompatibility
@mattt I tried this approach but it didn't work, which is why I went all in on vendoring. Although in my tests Pydantic was still pinned at <2, but I replaced all imports with the try..catch v1 thing. When I ran it on a model that installed Pydantic 2 it still failed. It might work when Pydantic isn't pinned but it's worth trying to build Cog models that include pydantic~=1 and pydantic~=2 on top of these changes, to make sure it still works.
@andreasjansson Do you recall what errors you saw? It'd be great to have a minimal example that reproduces this problem.
Digging into this some more, I found this discussion and this issue about problems arising from FastAPI >= 0.100.0 attempting to load v2 Pydantic when the v1 shim is loaded. From there, I found this PR which makes the necessary changes to get all of that working. I forked the repo for that PR to replicate/fastapi, rebased against official FastAPI, and updated this PR to use fastapi from our fork.
I got tests passing locally (aside from hypothesis timeouts and flakes), but can't resolve failures in CI.
=========================================================== short test summary info ============================================================
FAILED python/tests/server/test_worker.py::test_fatalworkerexception_from_irrecoverable_failures[killed_in_predict-payloads1] - hypothesis.errors.DeadlineExceeded: Test took 1124.96ms, which exceeds the deadline of 1000.00ms
FAILED python/tests/server/test_worker.py::test_output[complex_output-payloads2-<lambda>] - hypothesis.errors.DeadlineExceeded: Test took 1195.50ms, which exceeds the deadline of 1000.00ms
FAILED python/tests/server/test_worker.py::test_output[hello_world-payloads0-<lambda>] - hypothesis.errors.Flaky: Hypothesis test_output(data=data(...), name='hello_world', payloads={'name': sampled_from(['John', 'Barry', 'Elspe...
FAILED python/tests/server/test_worker.py::test_output[count_up-payloads1-<lambda>] - hypothesis.errors.Flaky: Hypothesis test_output(data=data(...), name='count_up', payloads={'upto': integers(min_value=0, max_value=100)}, o...
requirements.txt
# This file was autogenerated by uv via the following command:
# uv pip compile pyproject.toml --extra=dev -o requirements.txt
anyio==4.3.0
# via
# httpx
# starlette
# watchfiles
attrs==23.2.0
# via hypothesis
black==24.4.0
build==1.2.1
certifi==2024.2.2
# via
# httpcore
# httpx
# requests
charset-normalizer==3.3.2
# via requests
click==8.1.7
# via
# black
# uvicorn
coverage==7.4.4
# via pytest-cov
execnet==2.1.1
# via pytest-xdist
fastapi @ git+https://github.com/replicate/fastapi.git@7be995a943adde5315cb557a9330ee4ff4137849
h11==0.14.0
# via
# httpcore
# uvicorn
httpcore==1.0.5
# via httpx
httptools==0.6.1
# via uvicorn
httpx==0.27.0
hypothesis==6.100.1
idna==3.7
# via
# anyio
# httpx
# requests
iniconfig==2.0.0
# via pytest
markupsafe==2.1.5
# via werkzeug
mypy-extensions==1.0.0
# via black
nodeenv==1.8.0
# via pyright
numpy==1.26.4
packaging==24.0
# via
# black
# build
# pytest
# pytest-rerunfailures
pathspec==0.12.1
# via black
pillow==10.3.0
platformdirs==4.2.0
# via black
pluggy==1.4.0
# via pytest
pydantic==1.10.15
# via fastapi
pyproject-hooks==1.0.0
# via build
pyright==1.1.347
pytest==8.1.1
# via
# pytest-cov
# pytest-rerunfailures
# pytest-xdist
pytest-cov==5.0.0
pytest-httpserver==1.0.10
pytest-rerunfailures==14.0
pytest-xdist==3.5.0
python-dotenv==1.0.1
# via uvicorn
pyyaml==6.0.1
# via
# responses
# uvicorn
requests==2.31.0
# via responses
responses==0.25.0
ruff==0.4.0
setuptools==69.5.1
# via nodeenv
sniffio==1.3.1
# via
# anyio
# httpx
sortedcontainers==2.4.0
# via hypothesis
starlette==0.37.2
# via fastapi
structlog==24.1.0
typing-extensions==4.11.0
# via
# fastapi
# pydantic
urllib3==2.2.1
# via
# requests
# responses
uvicorn==0.29.0
uvloop==0.19.0
# via uvicorn
watchfiles==0.21.0
# via uvicorn
websockets==12.0
# via uvicorn
werkzeug==3.0.2
# via pytest-httpserver
Upgrading to Pydantic 2 is a much easier task (#1625), and we should probably go with that. The hard part is figuring out compatibility with models built using older versions of Cog.
@mattt Yes the FastAPI issue is the same thing I ran into. When I realized we'd have to rely on a fork of FastAPI I went down the vendoring route.
I've had pretty good success with act for debugging CI tests locally, in cases where they work in my environment but fail in CI.
I'd also love to understand why vendoring is a bad idea.