Request for task to be added for converting stable-diffusion-xl-turbo to onnx format using optimum-cli

[theoctopusride]

Feature request

I tried using optimum-cli with task set as stable-diffusion-xl but the process was killed:

~/Downloads$ optimum-cli export onnx --model sdxl-turbo --task stable-diffusion-xl sdxl-turbo-onnx/
Framework not specified. Using pt to export to ONNX.
Keyword arguments {'subfolder': '', 'trust_remote_code': False} are not expected by StableDiffusionXLImg2ImgPipeline and will be ignored.
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████████████████| 7/7 [00:05<00:00,  1.34it/s]
Using framework PyTorch: 2.1.1+cu121
/home/apps/anaconda3/envs/stable_diffusion_env/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:66: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1 or self.sliding_window is not None:
/home/apps/anaconda3/envs/stable_diffusion_env/lib/python3.8/site-packages/transformers/modeling_attn_mask_utils.py:137: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if past_key_values_length > 0:
/home/apps/anaconda3/envs/stable_diffusion_env/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:273: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/home/apps/anaconda3/envs/stable_diffusion_env/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:281: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
/home/apps/anaconda3/envs/stable_diffusion_env/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py:313: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
/home/apps/.local/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:5857: UserWarning: Exporting aten::index operator of advanced indexing in opset 14 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  warnings.warn(
Killed

Motivation

I think a lot of people would like to use stable-diffusion-xl-turbo in onnx format.

Your contribution

not sure how to submit a PR,but I could help if needed. I'm pretty dumb though...

[echarlaix]

Hi @theoctopusride,

It looks like you're running out of memory during conversion (which is a memory-intensive step that can require up to multiple times your model size in space). If you are interested in the sdxl-turbo model, we recently added the ONNX model, that you should be able to load directly with the ORTStableDiffusionXLPipeline class