setup.py, OpenAPI support

[doctorpangloss]

This includes the setup.py changes:

• setup.py now works
• xformers is never installed by setup.py, because torch 2.0.1 makes it redundant and you specified scaled_dot_product_attention
• Makes installation work a variety of ways, including making other packages dependent on this one for e.g. plugins
• Fixes missing init.py issues (you addressed this)
• Fixes imports (you addressed this)
• Compatible with your existing scripts that rely on requirements.txt
• Fixes error in comfy/ldm/models/diffusion/ddim.py
• Fixes missing packages for other diffusers code in this repo

This doesn't mess up your CI/CD automations. Those will correctly install the latest / nightlies as you configured them too.

The extra version numbers in requirements.txt should eliminate the inevitable situation where users will try to install the package and get an incompatible or subtly broken version of a package.

Creates two new api endpoints documented in the openapi.yaml. These are ergonomic for common usage like batches or using from other applications.

[tildebyte]

This looks really good (worked well for me).

[doctorpangloss]

This is now tested on fairly complex setups except AMD DirectML on Windows. I will try that on my AMD Windows station later. Additionally, PyTorch 2.1+ nightlies are now the default for all accelerators I tested (Windows NVIDIA, Windows CPU, Linux NVIDIA, Linux AMD, macOS Apple Silicon). This leaves Windows AMD and macOS AMD to test.

[tildebyte]

PyTorch 2.1+ nightlies

Looks like for NVidia, I currently get torch@https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp311-cp311-win_amd64.whl (the url in setup.py is for torch 2.0.1 and CUDA 11.8) when I run pip install -e ., not 2.1+nightly (unless maybe there's a dependency which I'm not seeing which causes torch==2.1)

[doctorpangloss]

That's the error! I accidentally left out that commit, I'll tie up my other work shortly and tidy this up.

[mcmonkey4eva]

This PR makes a large amount of unrelated changes that makes it extremely difficult to review or accept, eg reformatting of unrelated code. Could you please resubmit only the core/relevant changes needed for your goal here?

(EDIT: Discussion happening about this on Matrix now)

[tildebyte]

Just to throw this out there - I've been running Comfy with this setup and (important part) Python 3.11 for weeks now without issue.

Unrelated, but, if I were a reviewer here, I'd at least ask that the API stuff be split into a separate PR - it's cool and all, but not really related to the bulk of what's here.

[doctorpangloss]

I'd at least ask that the API stuff be split into a separate PR

that can be done for sure. imo it's nice to deliver immediate ROI when you take your long term ROI packaging bitter medicine

[oxc]

It would be nice to be able to select the target system manually, not only through auto detection. One might want to provide a list of prebuilt packages, on a build machine that has none of the GPUs it is building for.

[doctorpangloss]

One might want to provide a list of prebuilt packages

that's what pip's dependency management is. In any case, you can always install your preferred version of torch, and everything will just work.

[Chaoses-Ib]

The readme about authoring custom nodes uses setup.py to store project metadata. How about storing the metadata in pyproject.toml? It's more general, one can choose other backends like Hatching, PDM and Flit by simply changing build-system.

The entry points can be specified like

[project.entry-points."comfyui.custom_nodes"]
mypackage = "mypackage_custom_nodes"