Stable-Diffusion-Webui-Civitai-Helper Failed to download model image. And other sample image issues.

Failed to download model image. And other sample image issues.

Open Taithrah opened this issue 6 months ago • 0 comments

Have you read document?

yes

Have you checked console log window's msg?

yes

Describe Issue

Some models refuse to download sample images.
Sample images are small regardless of "Download Max Size Preview" checked on.
Images are download as jpeg. If you look at the header, they are png.

Example of a model that successfully downloads all images: https://civitai.com/api/v1/models/616858 Example of a model that fail to downloads images: https://civitai.com/api/v1/models/616848

Some things i have noticed on civitai

When looking at one of the models here https://civitai.com/api/v1/models/616848 I noticed image URLs are displayed like. https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/38603286-757b-4a36-b04e-48feae654c13/width=450/22385518.jpeg

width=450 should be at least width=768 however omitting it entirely like this will also download the full size image. https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/38603286-757b-4a36-b04e-48feae654c13/22385518.jpeg

When looking at the response, it shows content-type: image/png You can also change the jpeg extension to png like the following. https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/38603286-757b-4a36-b04e-48feae654c13/22385518.png

you can probably leverage import filetype to determine the correct filetype.

Console log's msg or screenshot for function issue

CHv1.8.10: Run get_model_id_from_url
CHv1.8.10: Getting model info for: 616848
CHv1.8.10: Request model info from civitai: 616848
CHv1.8.10: Get subfolder for: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora
CHv1.8.10: Searching local model by version id
CHv1.8.10: folder: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\
CHv1.8.10: model_ids: {'model': 616848, 'version': 689579}
CHv1.8.10: Start downloading from: https://civitai.com/api/download/models/689579
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.safetensors
CHv1.8.10: File size: 228457932 (217.87M)
CHv1.8.10: Downloading to temp file: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.safetensors.downloading
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 218M/218M [00:02<00:00, 96.3MiB/s]
CHv1.8.10: File Downloaded to: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.safetensors
CHv1.8.10: Request version info from civitai
CHv1.8.10: Fetching Parent Model Information
CHv1.8.10: Request model info from civitai: 616848
CHv1.8.10: No existing model info.
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/38603286-757b-4a36-b04e-48feae654c13/width=450/22385518.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: File size: 371720 (363.01K)
CHv1.8.10: Downloading to temp file: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg.downloading
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 363k/363k [00:00<00:00, 62.0MiB/s]
CHv1.8.10: File Downloaded to: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/8d355991-2305-4cee-8345-cc4f02cd2e24/width=450/22385513.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/8d355991-2305-4cee-8345-cc4f02cd2e24/width: 450/22385513.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/a413dcdc-ccfc-4509-86cd-2ffdf89cd7b0/width=450/22385515.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/a413dcdc-ccfc-4509-86cd-2ffdf89cd7b0/width: 450/22385515.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/c25007da-cf27-4040-ac53-568b4cc56171/width=450/22385516.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/c25007da-cf27-4040-ac53-568b4cc56171/width: 450/22385516.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/733768f7-181c-405e-9c22-3b453b15068f/width=450/22385512.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/733768f7-181c-405e-9c22-3b453b15068f/width: 450/22385512.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/51e8fcef-bdc6-44d2-b16f-6ecead73cbd0/width=450/22385520.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/51e8fcef-bdc6-44d2-b16f-6ecead73cbd0/width: 450/22385520.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/743810fe-28c0-4a1f-a069-b96ac3e32a63/width=450/22385521.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/743810fe-28c0-4a1f-a069-b96ac3e32a63/width: 450/22385521.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/08eeeb11-442e-4615-988b-c84ea7aca043/width=450/22385524.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/08eeeb11-442e-4615-988b-c84ea7aca043/width: 450/22385524.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/e8829f12-e90b-4461-b680-6461c0a0e0fc/width=450/22385525.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/e8829f12-e90b-4461-b680-6461c0a0e0fc/width: 450/22385525.jpeg
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/43083141-46b3-40e4-80ef-fb430f745359/width=450/22385523.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.example.0.jpeg
CHv1.8.10: Download failed.
   Failed to download model image.
   Download url: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/43083141-46b3-40e4-80ef-fb430f745359/width: 450/22385523.jpeg
CHv1.8.10: Write model civitai info to file: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.civitai.info
CHv1.8.10: Write model webui info to file: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.json
CHv1.8.10: Downloading model image.
CHv1.8.10: Checking preview image for model: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.safetensors
CHv1.8.10: Start downloading from: https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/08eeeb11-442e-4615-988b-c84ea7aca043/width=768/22385524.jpeg
CHv1.8.10: Target file path: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.preview.png
CHv1.8.10: File size: 784295 (765.91K)
CHv1.8.10: Downloading to temp file: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.preview.png.downloading
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 766k/766k [00:00<00:00, 64.8MiB/s]
CHv1.8.10: File Downloaded to: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.preview.png
CHv1.8.10: Done. Downloaded to: D:\projects\stable-diffusion\stable-diffusion-webui\models\Lora\[Majenta Rose] 'Succubus' Artstyle Dataset Revision PonyXL.safetensors

Extra

I wrote a script that helps me batch download sample images at full size and the correct filetype to help me temporarily work around this issue. API key is fetched from config.json

import os
import json
import re
import filetype
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
from requests_futures.sessions import FuturesSession

def download_image(url, file_path, api_key):
    session = FuturesSession()
    response = session.get(url, headers={"Authorization": f"Bearer {api_key}"}).result()
    if response.status_code == 200:
        # Determine the correct extension
        kind = filetype.guess(response.content)
        if kind is None:
            # Default to original extension if type cannot be determined
            file_extension = url.split(".")[-1]
        else:
            file_extension = kind.extension
        
        # Save the image with the correct extension
        file_path = re.sub(r'\.\w+$', f'.{file_extension}', file_path)
        
        with open(file_path, 'wb') as file:
            file.write(response.content)
        print(f"Downloaded and saved {file_path}")
    else:
        print(f"Failed to download {url}, status code: {response.status_code}")

def download_images(model_id, api_key):
    # Create the subfolder named after the model_id
    output_folder = model_id
    if not os.path.exists(output_folder):
        os.makedirs(output_folder)

    # Construct the URL
    json_url = f"https://civitai.com/api/v1/models/{model_id}?token={api_key}"

    # Fetch the JSON data
    response = requests.get(json_url)
    data = response.json()

    # Iterate over the images and prepare download tasks
    suffix = 1
    tasks = []
    for model_version in data["modelVersions"]:
        # Get the file name from the files section
        file_name_base = model_version["files"][0]["name"].rsplit('.', 1)[0].replace(" ", "_")
        for image in model_version["images"]:
            url = image["url"]
            # Remove the width parameter using regex
            url = re.sub(r'/width=\d+/', '/', url)
            # Create a filename with the base name and suffix
            file_name = f"{file_name_base}_{suffix}.tmp"  # Temporary extension to be replaced later
            file_path = os.path.join(output_folder, file_name)
            # Add the task to the list
            tasks.append((url, file_path, api_key))
            suffix += 1

    # Use ThreadPoolExecutor to download images concurrently
    with ThreadPoolExecutor(max_workers=6) as executor:
        future_to_url = {executor.submit(download_image, *task): task for task in tasks}
        for future in as_completed(future_to_url):
            task = future_to_url[future]
            try:
                future.result()
            except Exception as exc:
                print(f"Error downloading {task[0]}: {exc}")

    print(f"Download completed for model ID {model_id}.")

# Load the API key from config.json or config.txt
with open('config.json') as config_file:
    config = json.load(config_file)
api_key = config.get('api_key')

if not api_key:
    raise ValueError("API key not found in config.json")

while True:
    # Prompt the user for model IDs
    model_ids = input("Enter the model IDs separated by spaces: ").split()

    # Download images for each model ID
    for model_id in model_ids:
        download_images(model_id, api_key)

    # Ask if the user wants to enter another model ID
    again = input("Do you want to enter another model ID? (yes/no): ").strip().lower()
    if again != 'yes':
        break

Aug 06 '24 02:08 Taithrah

Stable-Diffusion-Webui-Civitai-Helper Stable-Diffusion-Webui-Civitai-Helper copied to clipboard

Failed to download model image. And other sample image issues.

Have you read document?

Have you checked console log window's msg?

Describe Issue

Some things i have noticed on civitai

Console log's msg or screenshot for function issue

Extra

Stable-Diffusion-Webui-Civitai-Helper
Stable-Diffusion-Webui-Civitai-Helper copied to clipboard