Updated stable_diffusion_tensorrt_txt2img.py

[Bhavay-2001]

What does this PR do?

Fixes #6984

Before submitting

• [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [x] Did you read the contributor guideline?
• [x] Did you read our philosophy doc (important for complex PRs)?
• [x] Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
• [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

[Bhavay-2001]

@standardAI @yiyixuxu

[yiyixuxu]

@standardAI can you do a review? thanks!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[tolgacangoz]

OK, let's continue. So far so good. When I run the current implementation by using the test script from the previous PR page I encounter an attribute error. Could you work on it? I would like to suggest a workflow: Try to work until no error is seen. If you cannot solve an error then ask ChatGPT and/or Gemini. If you don't get a possible solution within 1 hour then could you ask here?

[Bhavay-2001]

Alright. I will work with that test example.

[Bhavay-2001]

Hi @standardAI, I will continue now. I am out of shape. Can you pls let me know how to proceed? Thanks

[Bhavay-2001]

import torch
from diffusers import DDIMScheduler
from diffusers.pipelines import DiffusionPipeline

Use the DDIMScheduler scheduler here instead
scheduler = DDIMScheduler.from_pretrained("stabilityai/stable-diffusion-2-1",
                                            subfolder="scheduler")

pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1",
                                                custom_pipeline="/workspace/diffusers/examples/community/stable_diffusion_tensorrt_txt2img.py",
                                                revision='fp16',
                                                torch_dtype=torch.float16,
                                                scheduler=scheduler,)

# re-use cached folder to save ONNX models and TensorRT Engines
pipe.set_cached_folder("stabilityai/stable-diffusion-2-1", revision='fp16',)

pipe = pipe.to("cuda")

prompt = "a beautiful photograph of Mt. Fuji during cherry blossom"
image = pipe(prompt).images[0]
image.save('tensorrt_mt_fuji.png')

Should I try to run this script on google colab with the stable_diffusion_tensorrt_txt2img.py from the changes I have made?

[surajp92]

@Bhavay-2001 let me know if you want to work together to finish this. I was reviewing this PR and I got the below error

ValueError: Pipeline <class 'diffusers_modules.local.stable_diffusion_tensorrt_txt2img.TensorRTStableDiffusionPipeline'> expected {'tokenizer', 'text_encoder', 'vae', 'unet', 'safety_checker', 'feature_extractor', 'scheduler', 'image_encoder'}, but only {'tokenizer', 'text_encoder', 'vae', 'unet', 'scheduler'} were passed.

@standardAI you mentioned that you got an attributeerror while trying to run the test script. Did you set the custom_pipeline path to the new file ( ~/stable_diffusion_tensorrt_txt2img.py)? I got the AttributeError related to tensorrt package while using the stable_diffusion_tensorrt_txt2img pipeline, i.e., the one based on the StableDiffusionPipeline

[tolgacangoz]

@surajp92 Tbh, I don't remember what I did. Now, I am seeing a similar ValueError:

ValueError: Pipeline <class 'diffusers_modules.local.stable_diffusion_tensorrt_txt2img.TensorRTStableDiffusionPipeline'> expected {'tokenizer', 'unet', 'scheduler', 'safety_checker', 'vae', 'feature_extractor', 'text_encoder'}, but only {'tokenizer', 'unet', 'scheduler', 'vae', 'feature_extractor', 'text_encoder'} were passed.

@Bhavay-2001, we can use it for testing. Just replace revision with variant to avoid using a deprecated parameter. If you look around other pipelines' initializer/constructor functions, you will see something about the two additional parameters in the ValueError message. Could you try to take references from them? Isn't diffusers.yml file irrelevant here? Lastly, the corresponding README part needs to be updated.