Why didn't covert t5xxl into engine in sd3 pipeline?

[wendashi]

Thanks for the awesome demo🙏 But when I using the demo by: python3 demo_txt2img_sd3.py "A vibrant street wall covered in colorful graffiti, the centerpiece spells "SD3 MEDIUM", in a storm of colors" --version sd3 --hf-token=$HF_TOKEN

found in /path/TensorRT/demo/Diffusion/stable_diffusion_3_pipeline.py

self.torch_fallback = dict(zip(model_names, [self.torch_inference or model_name in ('t5xxl') for model_name in model_names]))

Why just leave the t5xxl in torch? THX in advance.🙏

[moraxu]

CC @akhilg-nv or @asfiyab-nvidia who are more up to speed with demos

[asfiyab-nvidia]

The T5 model enablement in the TRT pipeline is ongoing and will be available in a few releases. I'll update here when we have a solid timeline

[wendashi]

Thanks for your prompt response! And I got another question is why the shape of t5_out is different with original diffusers SD3 pipeline?

in /path/TensorRT/demo/Diffusion/stable_diffusion_3_pipeline.py

t5_out, _ = encode_token_weights('t5xxl', tokens["t5xxl"])
# t5_out is torch.Size([1, 77, 4096])

but in original diffusers SD3 pipeline, the 't5_out'(prompt_embeds) is ([1, 256, 4096]

in /path/diffusers/src/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py

prompt_embeds = self.text_encoder_3(text_input_ids.to(device))[0]
# torch.Size([1, 256, 4096])