AUTOMATIC1111
[Bug]: This app keeps downgrading transformers installation, not needed (with solution)
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
What happened?
Whenever you start the webui, your system-wide pip transformer installation gets downgraded to version 4.19.2 for no reason, breaking other apps.
Steps to reproduce the problem
pip show transformers > Version: 4.24.0 python ./launch.py (wait some time and quit the app) pip show transformers > Version: 4.19.2
What should have happened?
the transformer installation, if it is newer than needed here, should not be touched. I have the most recent version 4.24.0 installed, and it works flawlessly with this stable-diffusion-webui. The versions have different caching of files, but both work out of the box with this app.
Commit where the problem happens
ba97889137672cf6c3a72ea3b77c48913211c9d3 (doesn't really matter)
What platforms do you use to access UI ?
Linux
What browsers do you use to access the UI ?
Mozilla Firefox
Command Line Arguments
python ./launch.py --autolaunch --gradio-img2img-tool color-sketch
Additional information, context and logs
Solution:
change requirements_version.txt from transformers=4.19.2 to transformers>=4.19.2 change requirements.txt from transformers=4.19.2 to transformers>=4.19.2
Whenever you start the webui, your system-wide pip transformer installation gets downgraded to version 4.19.2 for no reason, breaking other apps.
Using == in the requirements.txt ensures that you have a set of compatible modules. If you use >= and a new transformer breaks something then new users installing from requirements.txt will have a broken install. == is more stable.
I think your issue would be better solved by using a virtualenv. This gives your webui repo its own standalone Python installation with its own set of pip-installed libraries, separate from your system-wide Python. That way the webui can have all of its preferred versions without stomping on your other installations.
If you launch with webui.sh instead of python ./launch.py the webui.sh script will automatically create and put you in a python virtualenv ('venv' by default). If you prefer to run launch.py directly, and you have run webui.sh previously so that it has created the virtual environment then you could enter the venv like so:
~/stable-diffusion-webui$ . venv/bin/activate (that . is important)
(venv) ~/stable-diffusion-webui$ python ./launch.py
You can see your prompt changes to show that you are in the venv virtual environment. You can exit by typing deactivate. But while you are in it, when you run 'python' you are running webui's private version of it, so this run of 'launch' would not overwrite your system-wide install.
Hello cobryan05,
I wonder why people are afraid of transformer breaking anything, while 15 other packages like einops, lark, kornia, realesrgan come with no version requirements. This seems quite inconsistent to me.
As a compromise, what about using a line like transformers==4.* or at least transformers>=4.19.2,<=4.24.0. If future transformer updates broke anything, it would likely get noticed very quickly by the community and it would probably benefit us to incorporate the change and upgrade to the newest version.
P.S. I don't want to introduce virtual environments for one single package that is actually not even incompatible.
while 15 other packages like einops, lark, kornia, realesrgan come with no version requirements. This seems quite inconsistent to me.
Inconsistent? Welcome to open source 😆
I see: einops==0.4.1, lark==1.1.2, kornia==0.6.7, realesrgan==0.3.0 in requirements-version.txt.
torch does not have a version listed in the requirements, but the webui.sh will cause it to install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
I'm not sure why there is a requirements.txt and a requirements-version.txt, but the latter specifies versions and is what launch.py uses.
As a compromise, what about using a line like transformers==4.*
I have no authority here to be making any compromises :) I was just explaining some generic rationale behind why setting specific version numbers is a 'good thing.' I used to not specify version numbers in my projects, or would use >=, and now many of my old python projects won't work anymore on fresh installs. I have learned the hard way to always specify exact version numbers.
P.S. I don't want to introduce virtual environments for one single package that is actually not even incompatible.
Well that may be an issue. By default, webui.sh does not have the issue you are running in to as virtual environments fix the issue and are automatically used by the launch scripts. If you want to run your local copy differently that sounds like you should make a local patch to your requirements.txt, rather than asking the repo to remove safeties around a potential time bomb.
If you made the edit locally and made a local commit, I believe in most cases a 'git pull' would rebase your local commit on top of any updates,, so you would maintain your local requirements.txt changes even when updating the repo.
Ah, I understand your point.
What about just bumping the version to 4.24.0 then. It has a better cache management and helps you delete the right files when you run low in memory after trying way too many models :-D Version 4.19.2 cache management is a mess.
(check your ~/.cache/huggingface/transformers dir to see the mess)
IMHO, as long as you are specifying a specific version then I think a version bump would be an acceptable solution/improvement. If you've ran the new version locally and haven't ran into issues you should try pushing a PR with your version updates. I have no authority here, but I'd think a one-line PR that fixes issues and doesn't cause any would be a slam dunk.
If this is still an issue and you believe this is actually a problem then open a PR to discuss resolving it. Closing.
