why won't any huggingface text2image models work?

[JosefAschauer]

Since I'm struggling with this since days and I know, that huggingface LLM's via their serverless inference api work just well same as text2image using openai's dall-e-3.

Problem: regardless what I configure in the litellm config - currently it's this but I tried nearly everything i could think of:

• model_name: huggingface/stable-diffusion-xl litellm_params: model: "huggingface/stabilityai/stable-diffusion-xl-base-1.0" api_base: "https://router.huggingface.co/hf-inference/models/stabilityai/stable-diffusion-xl-base-1.0" api_key: os.environ/HUGGINGFACE_API_KEY request_type: "post" request_body: inputs: "{{prompt}}" custom_llm_provider: "huggingface" # Explicitly set provider response_format: "base64" model_info: mode: image_generation

But the model info in liteLLM keeps showing mode: chat

with the bare minimum:

• model_name: huggingface/stable-diffusion-xl litellm_params: model: "huggingface/stabilityai/stable-diffusion-xl-base-1.0" api_base: "https://router.huggingface.co/hf-inference/models/stabilityai/stable-diffusion-xl-base-1.0" api_key: os.environ/HUGGINGFACE_API_KEY model_info: mode: image_generation

it's exactly the same.

In the logs I'm seeing a 200: INFO: 192.168.0.27:58640 - "POST /v1/images/generations HTTP/1.1" 200 OK

But it's never coming back with a valid response, probably since it expects a chat response. In the debug logs I also see that it's sending the api key.

itellm_1 | 08:09:24 - LiteLLM:DEBUG: utils.py:4579 - model_info: {'key': 'stabilityai/stable-diffusion-xl-base-1.0', 'max_tokens': None, 'max_input_tokens': None, 'max_output_tokens': None, 'input_cost_per_token': 0, 'output_cost_per_token': 0, 'litellm_provider': 'huggingface', 'mode': 'chat', 'supports_system_messages': None, 'supports_response_schema': None, 'supports_function_calling': None, 'supports_tool_choice': None, 'supports_assistant_prefill': None, 'supports_prompt_caching': None, 'supports_pdf_input': None}

and

litellm_1 | "request_id": "a44d8189-d1f0-411a-8632-58c96998cc4e", litellm_1 | "call_type": "aimage_generation", litellm_1 | "api_key": "88dc28d0f030c55ed4ab77xxxxxxxxxxxxxxxxxxxf778539800c9f1243fe6b4b", litellm_1 | "cache_hit": "None", litellm_1 | "startTime": "2025-03-14 08:09:24.237652+00:00", litellm_1 | "endTime": "2025-03-14 08:09:24.248845+00:00", litellm_1 | "completionStartTime": "2025-03-14 08:09:24.248845+00:00", litellm_1 | "model": "stabilityai/stable-diffusion-xl-base-1.0", litellm_1 | "user": "default_user_id", litellm_1 | "team_id": "", litellm_1 | "metadata": "{"user_api_key": "88dc28d0f030c55ed4ab77xxxxxxxxxxxxxxxxxxxf778539800c9f1243fe6b4b", "user_api_key_alias": null, "user_api_key_team_id": null, "user_api_key_org_id": null, "user_api_key_user_id": "default_user_id", "user_api_key_team_alias": null, "requester_ip_address": "", "applied_guardrails": [], "batch_models": null, "additional_usage_values": {"completion_tokens_details": null, "prompt_tokens_details": null}}",

To answer the obvious: Yes, I tested the huggingface endpoint directly in bash using CURL and the same api key and it gets me an image, so the problem lays somewhere either in my model config or litellm.

Any help would be just GREAT!