[Bug]: Cloned main, but webui can't complete installation: error pydantic

[IgorAherne]

Checklist

• [X] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [X] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

Initial launch fails every time, due to pydantic dependency error.

Steps to reproduce the problem

clone main launch webui-user.bat installation proceeds, but error is thrown before default neural net starts download. Press Any key to continue...

What should have happened?

installation should complete + download default neural network

What browsers do you use to access the UI ?

Other

Sysinfo

Windows 10 Webui UI doesn't launch because the webui-user.bat can't complete install. Was working fine few days ago, looks like pydantic did something recently.

Console logs

venv "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\Scripts\Python.exe"
Traceback (most recent call last):
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\launch.py", line 51, in <module>
    main()
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\launch.py", line 29, in main
    filename = launch_utils.dump_sysinfo()
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\modules\launch_utils.py", line 554, in dump_sysinfo
    from modules import sysinfo
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\modules\sysinfo.py", line 12, in <module>
    from modules import paths_internal, timer, shared, extensions, errors
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\modules\shared.py", line 4, in <module>
    import gradio as gr
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\gradio\__init__.py", line 3, in <module>
    import gradio.components as components
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\gradio\components\__init__.py", line 1, in <module>
    from gradio.components.annotated_image import AnnotatedImage
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\gradio\components\annotated_image.py", line 13, in <module>
    from gradio.components.base import IOComponent, _Keywords
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\gradio\components\base.py", line 20, in <module>
    from fastapi import UploadFile
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\fastapi\__init__.py", line 7, in <module>
    from .applications import FastAPI as FastAPI
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\fastapi\applications.py", line 15, in <module>
    from fastapi import routing
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\fastapi\routing.py", line 22, in <module>
    from fastapi import params
  File "C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\fastapi\params.py", line 4, in <module>
    from pydantic.fields import FieldInfo, Undefined
ImportError: cannot import name 'Undefined' from 'pydantic.fields' (C:\Users\Igor\Documents\StableProjectorz\stable-diffusion-webui-forge-main\venv\lib\site-packages\pydantic\fields.py)
Press any key to continue . . .

Additional information

No response

[IgorAherne]

@lllyasviel

[IgorAherne]

Quickly hacked it: Added pydantic==1.10.11 into the requirements.txt (it gets ignored though, likely some dependency wants new pydantic)

In the launch_utils.py function prepare_environment added the following indelicacy:

req_met = requirements_met(requirements_file)  #changed this line
if not req_met:                                                      #and this too.
    run_pip(f"install -r \"{requirements_file}\"", "requirements") 
    startup_timer.record("install requirements")

then at the very end of function, right before the --exit, I did this:

if not req_met:
    run_pip("install --force-reinstall pydantic==1.10.11", desc="pydantic")

if "--exit" in sys.argv:
    print("Exiting because of --exit argument")
    exit(0)

This allows it to get past the installation, and it starts to download the default neural network

Edit:
However, sometimes it would still complain and not work, after re-launching the webui. I "fixed" it by doing run_pip("install --upgrade pydantic==1.10.11", desc="pydantic") after the startup_timer.record("git version info") inside def prepare_environemnt. This forces pydantic to reinstall every webui launch, but at least it works :s

[CodeyNacke]

Had the same issue and the posted work-around fixed it.

[mechamuffin2000]

If you installed via the all-in-one package, run.bat will complete the install of Forge. However, trying to install via user-webui.bat always has this problem for some reason.

Tried to git clone and not prowess enough with python to understand the launch_utils.py workaround, hope it will be fixed soon.

[kaanyalova]

Quickly hacked it: Added pydantic==1.10.11 into the requirements.txt (it gets ignored though, likely some dependency wants new pydantic)

@IgorAherne

reqirements.txt is not used for requirements you should add it to requirements_versions.txt

[JuanAntBuit]

@kaanyalova @IgorAherne

I had this error and I can confirm that just adding pydantic==1.10.11 in requirements_versions.txt, without making changes to any other files, fixed it for me.

[DigiClau]

It worked without any issue last month, but today this error happened when I tried to reinstall. Has been anything changed since last month? There is no change history though.

[Jiejue233]

from my perspective, this bug happened because the pydantic package was rewritten in V2 so that the "undefined" feature no longer exists. refer to the pydantic github page

Pydantic V2 is a ground-up rewrite that offers many new features, performance improvements, and some breaking changes compared to Pydantic V1.

If you're using Pydantic V1 you may want to look at the pydantic V1.10 Documentation or, 1.10.X-fixes git branch. Pydantic V2 also ships with the latest version of Pydantic V1 built in so that you can incrementally upgrade your code base and projects: .from pydantic import v1 as pydantic_v1

[IgorAherne]

Quickly hacked it: Added pydantic==1.10.11 into the requirements.txt (it gets ignored though, likely some dependency wants new pydantic)

@IgorAherne

reqirements.txt is not used for requirements you should add it to requirements_versions.txt @kaanyalova

Nope, specifying just the version doesn't work for me:

Only if I do my hacky workaround, then it works

[mugenrei]

I wanted to build a new install dir but also got this issue, I'll post a pip freeze here of a working install, beware its not clean, has extensions dependencies amid the ones for webui hope it helps:

absl-py==2.1.0
accelerate==0.21.0
addict==2.4.0
aenum==3.1.15
aiofiles==23.2.1
aiohttp==3.9.3
aiosignal==1.3.1
albumentations==1.3.1
altair==5.2.0
antlr4-python3-runtime==4.9.3
anyio==3.7.1
astunparse==1.6.3
async-timeout==4.0.3
attrs==23.2.0
av==12.0.0
basicsr==1.4.2
beautifulsoup4==4.12.3
blendmodes==2022
cachetools==5.3.2
certifi==2024.2.2
cffi==1.16.0
chardet==5.2.0
charset-normalizer==3.3.2
clean-fid==0.1.35
click==8.1.7
clip==1.0
colorama==0.4.6
coloredlogs==15.0.1
colorlog==6.8.2
contourpy==1.2.0
cssselect2==0.7.0
cycler==0.12.1
Cython==3.0.8
datasets==2.17.1
deepdanbooru==1.0.2
deprecation==2.1.0
depth_anything @ https://github.com/huchenlei/Depth-Anything/releases/download/v1.0.0/depth_anything-2024.1.22.0-py2.py3-none-any.whl
diffusers==0.25.0
dill==0.3.8
diskcache==5.6.3
dynamicprompts==0.31.0
easydict==1.11
einops==0.4.1
embreex==2.17.7.post4
evaluate==0.4.1
exceptiongroup==1.2.0
facexlib==0.3.0
fake-useragent==1.4.0
fastapi==0.94.0
ffmpy==0.3.2
filelock==3.13.1
filterpy==1.4.5
flatbuffers==23.5.26
fonttools==4.48.1
frozenlist==1.4.1
fsspec==2023.10.0
ftfy==6.1.3
future==0.18.3
fvcore==0.1.5.post20221221
gast==0.5.4
gitdb==4.0.11
GitPython==3.1.32
google-auth==2.27.0
google-auth-oauthlib==1.2.0
google-pasta==0.2.0
gradio==3.41.2
gradio_client==0.5.0
grpcio==1.60.1
h11==0.12.0
h5py==3.10.0
handrefinerportable @ https://github.com/huchenlei/HandRefinerPortable/releases/download/v1.0.1/handrefinerportable-2024.2.12.0-py2.py3-none-any.whl
httpcore==0.15.0
httpx==0.24.1
huggingface-hub==0.20.3
humanfriendly==10.0
idna==3.6
imageio==2.34.0
imageio-ffmpeg==0.4.9
importlib-metadata==7.0.1
importlib-resources==6.1.1
inflection==0.5.1
insightface @ https://github.com/Gourieff/Assets/raw/main/Insightface/insightface-0.7.3-cp310-cp310-win_amd64.whl
iopath==0.1.9
Jinja2==3.1.3
joblib==1.3.2
jsonmerge==1.8.0
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
keras==2.15.0
kiwisolver==1.4.5
kornia==0.6.7
lark==1.1.2
lazy_loader==0.3
libclang==16.0.6
lightning-utilities==0.10.1
llama_cpp_python==0.2.26+cu121
llvmlite==0.42.0
lmdb==1.4.1
lxml==5.1.0
mapbox-earcut==1.0.1
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.8.2
mdurl==0.1.2
mediapipe==0.10.9
ml-dtypes==0.3.2
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
networkx==3.2.1
numba==0.59.0
numpy==1.26.2
oauthlib==3.2.2
omegaconf==2.2.3
onnx==1.15.0
onnxruntime==1.17.0
open-clip-torch==2.20.0
opencv-contrib-python==4.9.0.80
opencv-python==4.9.0.80
opencv-python-headless==4.9.0.80
opt-einsum==3.3.0
optimum==1.17.1
orjson==3.9.13
packaging==23.2
pandas==2.2.0
piexif==1.1.3
Pillow==9.5.0
platformdirs==4.2.0
portalocker==2.8.2
prettytable==3.9.0
protobuf==3.20.3
psutil==5.9.5
py-cpuinfo==9.0.0
pyarrow==15.0.0
pyarrow-hotfix==0.6
pyasn1==0.5.1
pyasn1-modules==0.3.0
pycollada==0.8
pycparser==2.21
pydantic==1.10.14
pydub==0.25.1
pyfunctional==1.4.3
Pygments==2.17.2
pyparsing==3.1.1
pyreadline3==3.4.1
PySocks==1.7.1
python-dateutil==2.8.2
python-dotenv==1.0.1
python-multipart==0.0.9
pytorch-lightning==1.9.4
pytz==2024.1
PyWavelets==1.5.0
pywin32==306
PyYAML==6.0.1
qudida==0.0.4
referencing==0.33.0
regex==2023.12.25
reportlab==4.1.0
requests==2.31.0
requests-oauthlib==1.3.1
resize-right==0.0.2
responses==0.18.0
rich==13.7.0
rpds-py==0.17.1
rsa==4.9
Rtree==1.2.0
safetensors==0.4.2
scikit-image==0.21.0
scikit-learn==1.4.0
scipy==1.12.0
seaborn==0.13.2
semantic-version==2.10.0
Send2Trash==1.8.2
sentencepiece==0.1.99
shapely==2.0.2
six==1.16.0
smmap==5.0.1
sniffio==1.3.0
sounddevice==0.4.6
soupsieve==2.5
spandrel==0.1.6
starlette==0.26.1
svg.path==6.3
svglib==1.5.1
sympy==1.12
tabulate==0.9.0
tb-nightly==2.16.0a20240212
tensorboard==2.15.2
tensorboard-data-server==0.7.2
tensorflow==2.15.1
tensorflow-estimator==2.15.0
tensorflow-intel==2.15.1
tensorflow-io-gcs-filesystem==0.31.0
termcolor==2.4.0
tf_keras-nightly==2.16.0.dev2024021210
thop==0.1.1.post2209072238
threadpoolctl==3.2.0
tifffile==2024.2.12
timm==0.9.12
tinycss2==1.2.1
tipo-kgen==0.0.10
tokenizers==0.13.3
tomesd==0.1.3
tomli==2.0.1
toolz==0.12.1
torch==2.1.2+cu121
torchdiffeq==0.2.3
torchmetrics==1.3.1
torchsde==0.2.6
torchvision==0.16.2+cu121
tqdm==4.66.2
trampoline==0.1.2
transformers==4.30.2
trimesh==4.1.3
typing_extensions==4.9.0
tzdata==2024.1
ultralytics==8.1.47
urllib3==2.2.0
uvicorn==0.27.1
vhacdx==0.0.5
wcwidth==0.2.13
webencodings==0.5.1
websockets==11.0.3
Werkzeug==3.0.1
wrapt==1.14.1
xxhash==3.4.1
yacs==0.1.8
yapf==0.40.2
yarl==1.9.4
zipp==3.17.0
ZipUnicode==1.1.1

[i486]

In addition to the issue stated above, another dependency conflict might occur when using the --forge-ref-a1111-home command with Forge's webui-user.bat to use an existing A1111-SDUI checkout.

Problem:

• Running Forge with A1111-SDUI checkout can lead to a failure and potentially break your existing vanilla SDUI installation due to dependency issues related to mediapipe, open-clip-torch, protobuf

Cause:

• The requirements.txt files in the forge_legacy_preprocessors sd_forge_controlnet folders lack a specific version for the mediapipe dependency. As a result Forge will install the latest mediapipe (currently 0.10.14), which triggers the uninstallation of protobuf 3.xx and installation of protobuf 4.25.3.
• However, open-clip-torch 2.20.0 requires protobuf<4 (aka 3.xx), resulting in a broken SDUI.

Solution 1: Upgrade open-clip-torch:

pip install open-clip-torch==2.24.0

This upgrades open-clip-torch to a version that doesn't require protobuf<4.

Solution 2: Downgrade mediapipe (and reinstall protobuf 3.xx):

pip install open-clip-torch 2.20.0  # This will also install protobuf 3.xx
pip install mediapipe==0.10.11

This downgrades mediapipe to a version compatible with protobuf 3.xx.

[IgorAherne]

Doesn't occur with the latest main (September 2024)