stable-diffusion-pytorch icon indicating copy to clipboard operation
stable-diffusion-pytorch copied to clipboard

Support loading original stable diffusion checkpoints

Open rnxyfvls opened this issue 1 year ago • 2 comments

  • Add checkpoint_loader based on code from https://github.com/kjsman/stable-diffusion-pytorch/issues/7#issuecomment-1426839447
  • Add example program

rnxyfvls avatar Mar 03 '24 06:03 rnxyfvls

Thank you for your contribution! Some thoughts:

  • It seems these patches have largely three points:
    • Converting the original SD checkpoint to the model which is compatible to this repository
    • Converting the safetensor model to the compatible model using the first component
    • The minimal demo Python code which "just works" with the given prompt, without using Jupyter Notebook.
  • I like all of them and want to ship them all, but how about separate them and name them clearer? I'm thinking of creating a stable_diffusion_pytorch/convert_from_sdmodel.py (which does same thing with checkpoint_loader.py), safetensor/convert_model.py (convert the safetensor model and saves to somewhere), and safetensor/generate_image.py (convert the safetensor model on-demand and generate image; do the same thing with demo_loadmodel)
  • I think using the official Stable Diffusion v1.5 safetensor model would be better for this general demo. I also like custom models such as Anything :) but I also want to keep things more official and academical in this repository.
  • The checkpoint converting code is written by @treeform I guess? We would need their approval to distribute this code as MIT license.

kjsman avatar Mar 03 '24 12:03 kjsman

  • Move conversion code to stable_diffusion_pytorch/convert_from_sdmodel.py: done
  • Change demo to stock SD 1.5 base model: done
  • convert_model.py: done, I have put it in the root directory though as it needs imports from stable_diffusion_pytorch/, and main programs can't import from parent directory
  • Demo program demo_sdmodel_generate_image.py: also in root directory for the above reason
  • Code license: I have asked treeform on issue #7 , if I can't get a response from him I will rewrite that part from scratch

rnxyfvls avatar Mar 04 '24 08:03 rnxyfvls