Diffusers on SageMaker

[AmanKishore]

Is your feature request related to a problem? Please describe. I'd like to run inference with stable diffusion through an API request!

Describe the solution you'd like Either an Inference API or Sagemaker integration!

Additional context Similar to https://huggingface.co/inference-endpoints or https://github.com/aws/sagemaker-huggingface-inference-toolkit

[MajorTal]

+1 I'd like to run pipeline.deploy() which is the standard HuggingFace way of deploying.

[patrickvonplaten]

cc @philschmid - could this make sense?

[philschmid]

@AmanKishore you can already use diffusers on sagemaker with the Hugging Face inference toolkit and a inference.py + a requirements.txt. Here is an example on how to do this: https://github.com/huggingface/notebooks/blob/main/sagemaker/17_custom_inference_script/sagemaker-notebook.ipynb

[MajorTal]

How do I add "diffusers" there? I'm guessing that's what the requirements.txt is for? It's not in the example... Also, how do I download the data? I need to plug in my access token somehow, just git+lfs is not enough. Where do I send the pipeline to Cuda? This should happen in the model_fn()? Thank you!

[philschmid]

How do I add "diffusers" there? I'm guessing that's what the requirements.txt is for? It's not in the example...

You add the requirements.txt here: https://github.com/huggingface/notebooks/tree/main/sagemaker/17_custom_inference_script/code

Also, how do I download the data?

Not sure what you mean.

I need to plug in my access token somehow, just git+lfs is not enough.

add it to the inference.py? or load it as a SSM secret?

Where do I send the pipeline to Cuda? This should happen in the model_fn()?

Yes

[MajorTal]

I managed to get it to work - thanks for the tips! I still think it should be built in to the general HuggingFace library, and deployed with a simple pipe.deploy().

[julien-c]

cc @jeffboudier for visibility

[Kitenite]

Came across the same problem, this is big help. Thank you both :)

[merfi]

@MajorTal can you share your solution, I'm facing the same issue

[DanielKneipp]

I've made this repo with everything needed to provision a sagemaker endpoint with API gateway (for public access) of a stable diffusion using huggingface toolkit. I still need to prepare the readme with all the docs but the code is there

[merfi]

awsome! Thanks, @DanielKneipp, that's very helpful

[baldesco]

How are you able to use the diffusers library? I added it to code/requirements.txt, and I'm getting this error:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message "{
  "code": 400,
  "type": "InternalServerException",
  "message": "No module named \u0027diffusers\u0027"
}

I'm running this in a Sagemaker notebook instance

[DanielKneipp]

readme is there now! 🎉 @baldesco I'd download the zip file with the model just to be double-sure that the code/requirements.txt is there with the diffusers dependency set.

[philschmid]

Hey, I also created a nice end-to-end example on how to deploy Stable Diffusion on to Amazon SageMaker: https://www.philschmid.de/sagemaker-stable-diffusion

It covers:

1. Create Stable Diffusion inference script
2. Create SageMaker model.tar.gz artifact
3. Deploy the model to Amazon SageMaker
4. Generate images using the deployed model

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.