Update to python 3.12

[cootshk]

Adds python ~~3.11~~ 3.12 support. See https://github.com/yaml/pyyaml/compare/6.0...6.0.1

[mashb1t]

@cootshk is this the only package you noticed for Fooocus to be compatible with Python 3.11?

[cootshk]

@mashb1t I updated all of the packages I needed to update for the project to run (MacOS, python 3.11.4 venv)

Testing on a newly created venv right now

[cootshk]

Seems to work in a new virtual environment from my small amount of testing (also it should work with python 3.12 if you compile scipy)

[mashb1t]

related to the python 3.12 compatibility issue: https://github.com/lllyasviel/Fooocus/issues/2938

reminder @ self for adding a hint when releasing milestone https://github.com/lllyasviel/Fooocus/milestone/4: for anyone else reading this who is using the zip file: run ..\python_embeded\python.exe -m pip install -r .\requirements_versions.txt from within the Fooocus folder to upgrade the packages.

[blckbx]

@cootshk

Pillow=10.0.0

For Python 3.12 compatibility, Pillow needs to be upped to minimum version 10.0.0 (tested on Ubuntu 24.04). transformers may be strictly pinned to 4.40.

[cootshk]

Tested on my own linux system (NixOS 23.11, RTX 3060) - everything works as intended my shell.nix file (has external libraries):

# Run with `nix-shell cuda-fhs.nix`
{ pkgs ? import <nixpkgs> {} }:
(pkgs.buildFHSUserEnv {
  name = "cuda-env";
  targetPkgs = pkgs: with pkgs; [ 
    git
    gitRepo
    gnupg
    autoconf
    curl
    procps
    gnumake
    util-linux
    m4
    gperf
    unzip
    cudatoolkit
    linuxPackages.nvidia_x11
    libGLU libGL
    xorg.libXi xorg.libXmu freeglut
    xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib 
    ncurses5
    stdenv.cc
    binutils
    nvidia-docker
    python311Full
    steam-run
  ];
  multiPkgs = pkgs: with pkgs; [ zlib ];
  runScript = "bash";
  profile = ''
    export CUDA_PATH=${pkgs.cudatoolkit}
    # export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib
    export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
    export EXTRA_CCFLAGS="-I/usr/include"
  '';
}).env

[cootshk]

If you want me to update everything for 3.12 I can, as 3.12 is what ubuntu 24.04 LTS will be shipping with

[mashb1t]

@cootshk thanks for the update! In general Fooocus should be updated to python3.12, also incl. the embedded version of python in the zip file, which is in the backlog. So feel free to update to 3.12!

[cootshk]

@mashb1t done!

[cootshk]

nevermind - github doesn't want to cooperate

[cootshk]

There we go - now it's ready to merge

[cootshk]

also I'd recommend making a branch for python 3.10, as the Cuda docker image and Colab aren't updated yet

[mashb1t]

@cootshk i'm having various warnings using python 3.12 as many SyntaxWarning are reported from 3rd party packages, see https://docs.python.org/3/whatsnew/3.12.html#other-language-changes. As we can't control the other packages i fear that users not knowing Fooocus are going to flood the discussion section.

A backslash-character pair that is not a valid escape sequence now generates a SyntaxWarning, instead of DeprecationWarning. For example, re.compile("\d+.\d+") now emits a SyntaxWarning ("\d" is an invalid escape sequence, use raw strings for regular expression: re.compile(r"\d+.\d+")). In a future Python version, SyntaxError will eventually be raised, instead of SyntaxWarning. (Contributed by Victor Stinner in gh-98401.)

Furthermore the default python version of Ubuntu (dockerhub) and Colab still use 3.10 as default, so we'd need to uninstall 3.10 and install 3.12 everywhere.

Another point is that users already having installed 3.10 would have to upgrade if packages are not compatible anymore in the future.

This currently leads me to the conclusion that staying with 3.10 for now as default is better.

For everybody wanting to run 3.12 please adjust the launch.py as shown here. You may still update to xformers 0.0.27 for Linux, feel free to adjust:

launch.py code

def prepare_environment():
    torch_index_url = os.environ.get('TORCH_INDEX_URL', "https://download.pytorch.org/whl/cu121")
    torch_command = os.environ.get('TORCH_COMMAND', f"pip install torch==2.1.0 torchvision==0.16.0 --extra-index-url {torch_index_url}")
    if platform.python_version().startswith("3.12"):
        torch_command = os.environ.get('TORCH_COMMAND', f"pip install torch==2.3.1 torchvision==0.18.1 --extra-index-url {torch_index_url}")
    requirements_file = os.environ.get('REQS_FILE', "requirements_versions.txt")

    print(f"Python {sys.version}")
    print(f"Fooocus version: {fooocus_version.version}")

    if REINSTALL_ALL or not is_installed("torch") or not is_installed("torchvision"):
        run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch", live=True)

    if TRY_INSTALL_XFORMERS:
        if REINSTALL_ALL or not is_installed("xformers"):
            xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.23')
            if platform.system() == "Windows":
                # support python 3.10 up until xformers 0.0.23 and pytorch 2.1.0+cu121
                if platform.python_version().startswith("3.10"):
                    run_pip(f"install -U -I --no-deps {xformers_package}", "xformers", live=True)
                elif platform.python_version().startswith("3.12"):
                    xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.27')
                    run_pip(f"install -U -I --no-deps {xformers_package}", "xformers", live=True)
                else:
                    print("Installation of xformers is not supported in this version of Python.")
                    print(
                        "You can also check this and build manually: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Xformers#building-xformers-on-windows-by-duckness")
                    if not is_installed("xformers"):
                        exit(0)
            elif platform.system() == "Linux":
                run_pip(f"install -U -I --no-deps {xformers_package}", "xformers")