[Bug]: Control net IP Adapter is not working. SDXL and 1.5

[Dervlex]

Checklist

• [X] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [ ] The issue has not been reported before recently
• [X] The issue has been reported before but has not been fixed yet

What happened?

Control net IP Adapter isnt working.

Steps to reproduce the problem

Just try to render with control net ipadapter ^^

What should have happened?

Rendering

What browsers do you use to access the UI ?

No response

Sysinfo

{ "Platform": "Windows-10-10.0.19045-SP0", "Python": "3.10.9", "Version": "f0.0.12-latest-110-g15bb49e7", "Commit": "15bb49e761e837c0a3463a736762d11941ea69f7", "Script path": "C:\Users\dervl\stable-diffusion-webui-forge", "Data path": "C:\Users\dervl\stable-diffusion-webui-forge", "Extensions dir": "C:\Users\dervl\stable-diffusion-webui-forge\extensions", "Checksum": "295dd9907e2a267e5a06db025213a29b740b2f3b28e1b48ceab91a43bf9ddfc5", "Commandline": [ "launch.py", "--autolaunch", "--xformers" ], "Torch env info": { "torch_version": "2.1.2+cu121", "is_debug_build": "False", "cuda_compiled_version": "12.1", "gcc_version": null, "clang_version": null, "cmake_version": "version 3.27.0", "os": "Microsoft Windows 10 Pro", "libc_version": "N/A", "python_version": "3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] (64-bit runtime)", "python_platform": "Windows-10-10.0.19045-SP0", "is_cuda_available": "True", "cuda_runtime_version": "11.8.89\r", "cuda_module_loading": "LAZY", "nvidia_driver_version": "546.17", "nvidia_gpu_models": "GPU 0: NVIDIA GeForce RTX 3060", "cudnn_version": "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudnn_ops_train64_8.dll", "pip_version": "pip3", "pip_packages": [ "numpy==1.26.2", "open-clip-torch==2.20.0", "pytorch-lightning==1.9.4", "torch==2.1.2+cu121", "torchdiffeq==0.2.3", "torchmetrics==1.3.0.post0", "torchsde==0.2.6", "torchvision==0.16.2+cu121" ], "conda_packages": null, "hip_compiled_version": "N/A", "hip_runtime_version": "N/A", "miopen_runtime_version": "N/A", "caching_allocator_config": "", "is_xnnpack_available": "True", "cpu_info": [ "Architecture=9", "CurrentClockSpeed=3401", "DeviceID=CPU0", "Family=107", "L2CacheSize=4096", "L2CacheSpeed=", "Manufacturer=AuthenticAMD", "MaxClockSpeed=3401", "Name=AMD Ryzen 7 5800X3D 8-Core Processor ", "ProcessorType=3", "Revision=8450" ] }, "Exceptions": [ { "exception": "Error(s) in loading state_dict for ImageProjModel:\n\tsize mismatch for proj.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([3072, 1280]).", "traceback": [ [ "C:\Users\dervl\stable-diffusion-webui-forge\modules\scripts.py, line 830, process_before_every_sampling", "script.process_before_every_sampling(p, *script_args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py, line 115, decorate_context", "return func(*args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 533, process_before_every_sampling", "self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py, line 115, decorate_context", "return func(*args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 479, process_unit_before_every_sampling", "params.model.process_before_every_sampling(p, cond, mask, *args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_ipadapter\scripts\forge_ipadapter.py, line 147, process_before_every_sampling", "unet = opIPAdapterApply(" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py, line 724, apply_ipadapter", "self.ipadapter = IPAdapter(" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py, line 287, init", "self.image_proj_model.load_state_dict(ipadapter_model["image_proj"])" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py, line 2152, load_state_dict", "raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(" ] ] }, { "exception": "0", "traceback": [ [ "C:\Users\dervl\stable-diffusion-webui-forge\modules\scripts.py, line 854, postprocess_batch_list", "script.postprocess_batch_list(p, pp, *script_args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py, line 115, decorate_context", "return func(*args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 539, postprocess_batch_list", "self.process_unit_after_every_sampling(p, unit, self.current_params[i], pp, *args, **kwargs)" ] ] }, { "exception": "0", "traceback": [ [ "C:\Users\dervl\stable-diffusion-webui-forge\modules\scripts.py, line 830, process_before_every_sampling", "script.process_before_every_sampling(p, *script_args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py, line 115, decorate_context", "return func(*args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 533, process_before_every_sampling", "self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, **kwargs)" ] ] }, { "exception": "'StableDiffusionProcessingTxt2Img' object has no attribute 'resize_mode'", "traceback": [ [ "C:\Users\dervl\stable-diffusion-webui-forge\modules\scripts.py, line 798, process", "script.process(p, *script_args)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py, line 115, decorate_context", "return func(*args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 526, process", "self.process_unit_after_click_generate(p, unit, params, *args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py, line 115, decorate_context", "return func(args, **kwargs)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 279, process_unit_after_click_generate", "input_list, resize_mode = self.get_input_data(p, unit, preprocessor)" ], [ "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py, line 204, get_input_data", "resize_mode = external_code.resize_mode_from_value(p.resize_mode)" ] ] } ], "CPU": { "model": "AMD64 Family 25 Model 33 Stepping 2, AuthenticAMD", "count logical": 16, "count physical": 8 }, "RAM": { "total": "32GB", "used": "21GB", "free": "11GB" }, "Extensions": [ { "name": "a1111-sd-webui-lycoris", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\a1111-sd-webui-lycoris", "version": "df9c0093", "branch": "main", "remote": "https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris.git" }, { "name": "clip-interrogator-ext", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\clip-interrogator-ext", "version": "0f1a4591", "branch": "main", "remote": "https://github.com/pharmapsychotic/clip-interrogator-ext" }, { "name": "sd-webui-additional-networks", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\sd-webui-additional-networks", "version": "e9f3d622", "branch": "main", "remote": "https://github.com/kohya-ss/sd-webui-additional-networks.git" }, { "name": "ultimate-upscale-for-automatic1111", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\ultimate-upscale-for-automatic1111", "version": "728ffcec", "branch": "master", "remote": "https://github.com/Coyote-A/ultimate-upscale-for-automatic1111.git" } ], "Inactive extensions": [ { "name": "StyleSelectorXL", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\StyleSelectorXL", "version": "5a093f5a", "branch": "main", "remote": "https://github.com/ahgsql/StyleSelectorXL.git" }, { "name": "deforum-for-automatic1111-webui", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\deforum-for-automatic1111-webui", "version": "d3b00b3c", "branch": "automatic1111-webui", "remote": "https://github.com/deforum-art/deforum-for-automatic1111-webui.git" }, { "name": "multidiffusion-upscaler-for-automatic1111", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\multidiffusion-upscaler-for-automatic1111", "version": "fbb24736", "branch": "main", "remote": "https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111.git" }, { "name": "sd-dynamic-prompts", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\sd-dynamic-prompts", "version": "", "branch": null, "remote": null }, { "name": "sd-webui-boomer", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\sd-webui-boomer", "version": "85fcf1e5", "branch": "main", "remote": "https://github.com/Haoming02/sd-webui-boomer" }, { "name": "sd-webui-controlnet", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\sd-webui-controlnet", "version": "8bd8057c", "branch": "main", "remote": "https://github.com/Mikubill/sd-webui-controlnet.git" }, { "name": "sd-webui-pixelart", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\sd-webui-pixelart", "version": "23c7de46", "branch": "master", "remote": "https://github.com/mrreplicart/sd-webui-pixelart.git" }, { "name": "stable-diffusion-webui-anti-burn", "path": "C:\Users\dervl\stable-diffusion-webui-forge\extensions\stable-diffusion-webui-anti-burn", "version": "6f4bf0a7", "branch": "master", "remote": "https://github.com/klimaleksus/stable-diffusion-webui-anti-burn" } ], "Environment": { "COMMANDLINE_ARGS": " --autolaunch --xformers", "GRADIO_ANALYTICS_ENABLED": "False" }, "Config": { "ldsr_steps": 25, "ldsr_cached": true, "SCUNET_tile": 256, "SCUNET_tile_overlap": 8, "SWIN_tile": 192, "SWIN_tile_overlap": 8, "SWIN_torch_compile": false, "control_net_detectedmap_dir": "detected_maps", "control_net_models_path": "", "control_net_modules_path": "", "control_net_unit_count": 3, "control_net_model_cache_size": 5, "control_net_no_detectmap": false, "control_net_detectmap_autosaving": false, "control_net_allow_script_control": false, "control_net_sync_field_args": true, "controlnet_show_batch_images_in_ui": false, "controlnet_increment_seed_during_batch": false, "controlnet_disable_openpose_edit": false, "controlnet_disable_photopea_edit": false, "controlnet_photopea_warning": true, "controlnet_input_thumbnail": true, "styles_ui": "radio-buttons", "enable_styleselector_by_default": true, "deforum_keep_3d_models_in_vram": false, "deforum_enable_persistent_settings": false, "deforum_persistent_settings_path": "models/Deforum/deforum_persistent_settings.txt", "deforum_ffmpeg_location": "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\imageio_ffmpeg\binaries\ffmpeg-win64-v4.2.2.exe", "deforum_ffmpeg_crf": 17, "deforum_ffmpeg_preset": "slow", "deforum_debug_mode_enabled": false, "deforum_save_gen_info_as_srt": false, "deforum_embed_srt": false, "deforum_save_gen_info_as_srt_params": [ "Noise Schedule" ], "deforum_preview": "Off", "deforum_preview_interval_frames": 100, "additional_networks_extra_lora_path": "", "additional_networks_sort_models_by": "name", "additional_networks_reverse_sort_order": false, "additional_networks_model_name_filter": "", "additional_networks_xy_grid_model_metadata": "", "additional_networks_hash_thread_count": 1, "additional_networks_back_up_model_when_saving": true, "additional_networks_show_only_safetensors": false, "additional_networks_show_only_models_with_metadata": "disabled", "additional_networks_max_top_tags": 20, "additional_networks_max_dataset_folders": 20, "bmr_app_style": false, "bmr_rfr_style": true, "bmr_sav_style": true, "bmr_send_btn": true, "bmr_fit": true, "bmr_thumbs": true, "bmr_enlarge": false, "dp_ignore_whitespace": false, "dp_write_raw_template": false, "dp_write_prompts_to_file": false, "dp_parser_variant_start": "{", "dp_parser_variant_end": "}", "dp_parser_wildcard_wrap": "__", "dp_limit_jinja_prompts": false, "dp_auto_purge_cache": false, "dp_magicprompt_default_model": "Gustavosta/MagicPrompt-Stable-Diffusion", "dp_magicprompt_batch_size": 1, "sd_checkpoint_hash": "4199bcdd147e11328d5f3560301d5a7ab4ac7eeefbf49dc3eb663cb3e772b9ac", "sd_model_checkpoint": "revAnimated_v122.safetensors [4199bcdd14]", "disabled_extensions": [ "StyleSelectorXL", "deforum-for-automatic1111-webui", "multidiffusion-upscaler-for-automatic1111", "sd-dynamic-prompts", "sd-webui-boomer", "sd-webui-controlnet", "sd-webui-pixelart", "stable-diffusion-webui-anti-burn" ], "disable_all_extensions": "none", "sd_vae": "kl-f8-anime2.ckpt", "save_images_before_highres_fix": false, "outdir_samples": "", "outdir_txt2img_samples": "output\txt2img-images", "outdir_img2img_samples": "output\img2img-images", "outdir_extras_samples": "output\extras-images", "outdir_grids": "", "outdir_txt2img_grids": "output\txt2img-grids", "outdir_img2img_grids": "output\img2img-grids", "outdir_save": "log\images", "outdir_init_images": "output\init-images", "samples_save": true, "samples_format": "png", "samples_filename_pattern": "", "save_images_add_number": true, "save_images_replace_action": "Replace", "grid_save": true, "grid_format": "png", "grid_extended_filename": false, "grid_only_if_multiple": true, "grid_prevent_empty_spots": false, "grid_zip_filename_pattern": "", "n_rows": -1, "font": "", "grid_text_active_color": "#000000", "grid_text_inactive_color": "#999999", "grid_background_color": "#ffffff", "save_images_before_face_restoration": false, "save_images_before_color_correction": false, "save_mask": false, "save_mask_composite": false, "jpeg_quality": 80, "webp_lossless": false, "export_for_4chan": true, "img_downscale_threshold": 4.0, "target_side_length": 4000.0, "img_max_size_mp": 200.0, "use_original_name_batch": true, "use_upscaler_name_as_suffix": false, "save_selected_only": true, "save_init_img": false, "temp_dir": "", "clean_temp_dir_at_start": false, "save_incomplete_images": false, "notification_audio": true, "notification_volume": 100, "save_to_dirs": true, "grid_save_to_dirs": true, "use_save_to_dirs_for_ui": false, "directories_filename_pattern": "[date]", "directories_max_prompt_words": 8, "auto_backcompat": true, "use_old_emphasis_implementation": false, "use_old_karras_scheduler_sigmas": false, "no_dpmpp_sde_batch_determinism": false, "use_old_hires_fix_width_height": false, "dont_fix_second_order_samplers_schedule": false, "hires_fix_use_firstpass_conds": false, "use_old_scheduling": false, "use_downcasted_alpha_bar": false, "lora_functional": false, "extra_networks_show_hidden_directories": true, "extra_networks_dir_button_function": false, "extra_networks_hidden_models": "When searched", "extra_networks_default_multiplier": 1, "extra_networks_card_width": 0.0, "extra_networks_card_height": 0.0, "extra_networks_card_text_scale": 1, "extra_networks_card_show_desc": true, "extra_networks_card_order_field": "Path", "extra_networks_card_order": "Ascending", "extra_networks_tree_view_default_enabled": false, "extra_networks_add_text_separator": " ", "ui_extra_networks_tab_reorder": "", "textual_inversion_print_at_load": false, "textual_inversion_add_hashes_to_infotext": true, "sd_hypernetwork": "None", "sd_lora": "None", "lora_preferred_name": "Alias from file", "lora_add_hashes_to_infotext": true, "lora_show_all": false, "lora_hide_unknown_for_versions": [], "lora_in_memory_limit": 0, "lora_not_found_warning_console": false, "lora_not_found_gradio_warning": false, "sd_lyco": "None", "cross_attention_optimization": "Automatic", "s_min_uncond": 0, "token_merging_ratio": 0, "token_merging_ratio_img2img": 0, "token_merging_ratio_hr": 0, "pad_cond_uncond": false, "pad_cond_uncond_v0": false, "persistent_cond_cache": true, "batch_cond_uncond": true, "fp8_storage": "Disable", "cache_fp16_weight": false, "hide_samplers": [], "eta_ddim": 0, "eta_ancestral": 1, "ddim_discretize": "uniform", "s_churn": 0, "s_tmin": 0, "s_tmax": 0, "s_noise": 1, "k_sched_type": "Automatic", "sigma_min": 0.0, "sigma_max": 0.0, "rho": 0.0, "eta_noise_seed_delta": 0, "always_discard_next_to_last_sigma": false, "sgm_noise_multiplier": false, "uni_pc_variant": "bh1", "uni_pc_skip_type": "time_uniform", "uni_pc_order": 3, "uni_pc_lower_order_final": true, "sd_noise_schedule": "Default", "sd_checkpoints_limit": 2, "sd_checkpoints_keep_in_cpu": true, "sd_checkpoint_cache": 0, "sd_unet": "Automatic", "enable_quantization": false, "enable_emphasis": true, "enable_batch_seeds": true, "comma_padding_backtrack": 20, "upcast_attn": false, "randn_source": "GPU", "tiling": false, "hires_fix_refiner_pass": "second pass", "sdxl_crop_top": 0.0, "sdxl_crop_left": 0.0, "sdxl_refiner_low_aesthetic_score": 2.5, "sdxl_refiner_high_aesthetic_score": 6.0, "sd_vae_checkpoint_cache": 0, "sd_vae_overrides_per_model_preferences": true, "auto_vae_precision_bfloat16": false, "auto_vae_precision": true, "sd_vae_encode_method": "Full", "sd_vae_decode_method": "Full", "inpainting_mask_weight": 1, "initial_noise_multiplier": 1, "img2img_extra_noise": 0, "img2img_color_correction": false, "img2img_fix_steps": false, "img2img_background_color": "#ffffff", "img2img_editor_height": 720, "img2img_sketch_default_brush_color": "#ffffff", "img2img_inpaint_mask_brush_color": "#ffffff", "img2img_inpaint_sketch_default_brush_color": "#ffffff", "return_mask": false, "return_mask_composite": false, "img2img_batch_show_results_limit": 32, "overlay_inpaint": true, "return_grid": true, "do_not_show_images": false, "js_modal_lightbox": true, "js_modal_lightbox_initially_zoomed": true, "js_modal_lightbox_gamepad": false, "js_modal_lightbox_gamepad_repeat": 250.0, "sd_webui_modal_lightbox_icon_opacity": 1, "sd_webui_modal_lightbox_toolbar_opacity": 0.9, "gallery_height": "", "enable_pnginfo": true, "save_txt": false, "add_model_name_to_info": true, "add_model_hash_to_info": true, "add_vae_name_to_info": true, "add_vae_hash_to_info": true, "add_user_name_to_info": false, "add_version_to_infotext": true, "disable_weights_auto_swap": true, "infotext_skip_pasting": [], "infotext_styles": "Apply if any", "show_progressbar": true, "live_previews_enable": true, "live_previews_image_format": "png", "show_progress_grid": true, "show_progress_every_n_steps": 10, "show_progress_type": "Approx NN", "live_preview_allow_lowvram_full": false, "live_preview_content": "Prompt", "live_preview_refresh_period": 1000.0, "live_preview_fast_interrupt": false, "js_live_preview_in_modal_lightbox": false, "keyedit_precision_attention": 0.1, "keyedit_precision_extra": 0.05, "keyedit_delimiters": ".,\/!?%^;:{}=`~() ", "keyedit_delimiters_whitespace": [ "Tab", "Carriage Return", "Line Feed" ], "keyedit_move": true, "disable_token_counters": false, "extra_options_txt2img": [], "extra_options_img2img": [], "extra_options_cols": 1, "extra_options_accordion": false, "compact_prompt_box": false, "samplers_in_dropdown": true, "dimensions_and_batch_together": true, "sd_checkpoint_dropdown_use_short": false, "hires_fix_show_sampler": false, "hires_fix_show_prompts": false, "txt2img_settings_accordion": false, "img2img_settings_accordion": false, "interrupt_after_current": true, "localization": "None", "quicksettings_list": [ "sd_model_checkpoint", "sd_vae", "CLIP_stop_at_last_layers" ], "ui_tab_order": [], "hidden_tabs": [], "ui_reorder_list": [], "gradio_theme": "Default", "gradio_themes_cache": true, "show_progress_in_title": true, "send_seed": true, "send_size": true, "api_enable_requests": true, "api_forbid_local_requests": true, "api_useragent": "", "auto_launch_browser": "Local", "enable_console_prompts": false, "show_warnings": false, "show_gradio_deprecation_warnings": true, "memmon_poll_rate": 8, "samples_log_stdout": false, "multiple_tqdm": true, "enable_upscale_progressbar": true, "print_hypernet_extra": false, "list_hidden_files": true, "disable_mmap_load_safetensors": false, "hide_ldm_prints": true, "dump_stacks_on_signal": false, "face_restoration": false, "face_restoration_model": "CodeFormer", "code_former_weight": 0.5, "face_restoration_unload": false, "postprocessing_enable_in_main_ui": [], "postprocessing_operation_order": [], "upscaling_max_images_in_cache": 5, "postprocessing_existing_caption_action": "Ignore", "ESRGAN_tile": 192, "ESRGAN_tile_overlap": 8, "realesrgan_enabled_models": [ "R-ESRGAN 4x+", "R-ESRGAN 4x+ Anime6B" ], "dat_enabled_models": [ "DAT x2", "DAT x3", "DAT x4" ], "DAT_tile": 192, "DAT_tile_overlap": 8, "unload_models_when_training": false, "pin_memory": false, "save_optimizer_state": false, "save_training_settings_to_txt": true, "dataset_filename_word_regex": "", "dataset_filename_join_string": " ", "training_image_repeats_per_epoch": 1, "training_write_csv_every": 500.0, "training_xattention_optimizations": false, "training_enable_tensorboard": false, "training_tensorboard_save_images": false, "training_tensorboard_flush_every": 120.0, "canvas_hotkey_zoom": "Alt", "canvas_hotkey_adjust": "Ctrl", "canvas_hotkey_shrink_brush": "Q", "canvas_hotkey_grow_brush": "W", "canvas_hotkey_move": "F", "canvas_hotkey_fullscreen": "S", "canvas_hotkey_reset": "R", "canvas_hotkey_overlap": "O", "canvas_show_tooltip": true, "canvas_auto_expand": true, "canvas_blur_prompt": false, "canvas_disabled_functions": [ "Overlap" ], "interrogate_keep_models_in_memory": false, "interrogate_return_ranks": false, "interrogate_clip_num_beams": 1, "interrogate_clip_min_length": 24, "interrogate_clip_max_length": 48, "interrogate_clip_dict_limit": 1500.0, "interrogate_clip_skip_categories": [], "interrogate_deepbooru_score_threshold": 0.5, "deepbooru_sort_alpha": true, "deepbooru_use_spaces": true, "deepbooru_escape": true, "deepbooru_filter_tags": "", "CLIP_stop_at_last_layers": 1 }, "Startup": { "total": 17.077571630477905, "records": { "initial startup": 0.022020339965820312, "prepare environment/checks": 0.01601433753967285, "prepare environment/git version info": 0.0740666389465332, "prepare environment/torch GPU test": 1.8737008571624756, "prepare environment/clone repositores": 0.2472248077392578, "prepare environment/run extensions installers/a1111-sd-webui-lycoris": 0.0, "prepare environment/run extensions installers/clip-interrogator-ext": 3.7069759368896484, "prepare environment/run extensions installers/sd-webui-additional-networks": 0.0, "prepare environment/run extensions installers/ultimate-upscale-for-automatic1111": 0.0010013580322265625, "prepare environment/run extensions installers": 3.707977294921875, "prepare environment/run extensions_builtin installers/canvas-zoom-and-pan": 0.0, "prepare environment/run extensions_builtin installers/extra-options-section": 0.0, "prepare environment/run extensions_builtin installers/forge_legacy_preprocessors": 0.43939876556396484, "prepare environment/run extensions_builtin installers/forge_preprocessor_inpaint": 0.0, "prepare environment/run extensions_builtin installers/forge_preprocessor_marigold": 0.0, "prepare environment/run extensions_builtin installers/forge_preprocessor_normalbae": 0.0, "prepare environment/run extensions_builtin installers/forge_preprocessor_recolor": 0.0, "prepare environment/run extensions_builtin installers/forge_preprocessor_reference": 0.0, "prepare environment/run extensions_builtin installers/forge_preprocessor_revision": 0.0, "prepare environment/run extensions_builtin installers/forge_preprocessor_tile": 0.0010004043579101562, "prepare environment/run extensions_builtin installers/LDSR": 0.0, "prepare environment/run extensions_builtin installers/Lora": 0.0, "prepare environment/run extensions_builtin installers/mobile": 0.0, "prepare environment/run extensions_builtin installers/prompt-bracket-checker": 0.0, "prepare environment/run extensions_builtin installers/ScuNET": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_controlllite": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_controlnet": 0.4320640563964844, "prepare environment/run extensions_builtin installers/sd_forge_controlnet_example": 0.0010013580322265625, "prepare environment/run extensions_builtin installers/sd_forge_dynamic_thresholding": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_fooocus_inpaint": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_freeu": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_hypertile": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_ipadapter": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_kohya_hrfix": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_latent_modifier": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_multidiffusion": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_photomaker": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_sag": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_stylealign": 0.0010004043579101562, "prepare environment/run extensions_builtin installers/sd_forge_svd": 0.0, "prepare environment/run extensions_builtin installers/sd_forge_z123": 0.0, "prepare environment/run extensions_builtin installers/soft-inpainting": 0.0, "prepare environment/run extensions_builtin installers/SwinIR": 0.0, "prepare environment/run extensions_builtin installers": 0.8744649887084961, "prepare environment": 6.8314831256866455, "launcher": 0.0030024051666259766, "import torch": 4.05600905418396, "import gradio": 0.9678783416748047, "setup paths": 0.6315734386444092, "import ldm": 0.006005525588989258, "import sgm": 0.0, "initialize shared": 0.151137113571167, "other imports": 0.5114645957946777, "opts onchange": 0.0, "setup SD model": 0.0, "setup codeformer": 0.0010006427764892578, "setup gfpgan": 0.013011932373046875, "set samplers": 0.0, "list extensions": 0.007006168365478516, "restore config state file": 0.0, "list SD models": 0.012011051177978516, "list localizations": 0.0010008811950683594, "load scripts/custom_code.py": 0.012011051177978516, "load scripts/img2imgalt.py": 0.0, "load scripts/loopback.py": 0.0, "load scripts/outpainting_mk_2.py": 0.0010006427764892578, "load scripts/poor_mans_outpainting.py": 0.0, "load scripts/postprocessing_caption.py": 0.0, "load scripts/postprocessing_codeformer.py": 0.0, "load scripts/postprocessing_create_flipped_copies.py": 0.001001119613647461, "load scripts/postprocessing_focal_crop.py": 0.010009050369262695, "load scripts/postprocessing_gfpgan.py": 0.0, "load scripts/postprocessing_split_oversized.py": 0.0, "load scripts/postprocessing_upscale.py": 0.0010006427764892578, "load scripts/processing_autosized_crop.py": 0.0, "load scripts/prompt_matrix.py": 0.0, "load scripts/prompts_from_file.py": 0.0, "load scripts/sd_upscale.py": 0.001001119613647461, "load scripts/xyz_grid.py": 0.0010008811950683594, "load scripts/ldsr_model.py": 1.046950340270996, "load scripts/lora_script.py": 0.24822545051574707, "load scripts/scunet_model.py": 0.0420382022857666, "load scripts/swinir_model.py": 0.038034677505493164, "load scripts/hotkey_config.py": 0.0010006427764892578, "load scripts/extra_options_section.py": 0.0, "load scripts/legacy_preprocessors.py": 0.015013694763183594, "load scripts/preprocessor_inpaint.py": 0.014012575149536133, "load scripts/preprocessor_marigold.py": 0.17015457153320312, "load scripts/preprocessor_normalbae.py": 0.008007049560546875, "load scripts/preprocessor_recolor.py": 0.0, "load scripts/forge_reference.py": 0.0, "load scripts/preprocessor_revision.py": 0.0, "load scripts/preprocessor_tile.py": 0.001001119613647461, "load scripts/forge_controllllite.py": 0.014012575149536133, "load scripts/controlnet.py": 0.5145888328552246, "load scripts/xyz_grid_support.py": 0.0008871555328369141, "load scripts/sd_forge_controlnet_example.py": 0.0, "load scripts/forge_dynamic_thresholding.py": 0.005005359649658203, "load scripts/forge_fooocus_inpaint.py": 0.0, "load scripts/forge_freeu.py": 0.005004405975341797, "load scripts/forge_hypertile.py": 0.004003763198852539, "load scripts/forge_ipadapter.py": 0.008007287979125977, "load scripts/kohya_hrfix.py": 0.0040035247802734375, "load scripts/forge_latent_modifier.py": 0.005004405975341797, "load scripts/forge_multidiffusion.py": 0.010009050369262695, "load scripts/forge_photomaker.py": 0.004003763198852539, "load scripts/forge_sag.py": 0.0050046443939208984, "load scripts/forge_stylealign.py": 0.0, "load scripts/forge_svd.py": 0.04904460906982422, "load scripts/forge_z123.py": 0.04303908348083496, "load scripts/soft_inpainting.py": 0.0010008811950683594, "load scripts/lycoris_script.py": 0.31328439712524414, "load scripts/clip_interrogator_ext.py": 0.0990898609161377, "load scripts/additional_networks.py": 0.20218348503112793, "load scripts/addnet_xyz_grid_support.py": 0.0, "load scripts/lora_compvis.py": 0.0010013580322265625, "load scripts/metadata_editor.py": 0.0, "load scripts/model_util.py": 0.022239208221435547, "load scripts/safetensors_hack.py": 0.001001119613647461, "load scripts/util.py": 0.0, "load scripts/ultimate-upscale.py": 0.0010006427764892578, "load scripts/refiner.py": 0.0, "load scripts/seed.py": 0.0, "load scripts": 2.923882246017456, "load upscalers": 0.007006645202636719, "refresh VAE": 0.003002643585205078, "refresh textual inversion templates": 0.0, "scripts list_optimizers": 0.028025150299072266, "scripts list_unets": 0.0, "reload hypernetworks": 0.0, "initialize extra networks": 0.004004001617431641, "scripts before_ui_callback": 0.07606863975524902, "create ui": 0.6298038959503174, "gradio launch": 0.22220206260681152, "add APIs": 0.006005287170410156, "app_started_callback/lora_script.py": 0.0, "app_started_callback/controlnet.py": 0.0010006427764892578, "app_started_callback/lycoris_script.py": 0.0, "app_started_callback/clip_interrogator_ext.py": 0.0, "app_started_callback": 0.0010006427764892578 } }, "Packages": [ "absl-py==2.1.0", "accelerate==0.21.0", "addict==2.4.0", "aenum==3.1.15", "aiofiles==23.2.1", "aiohttp==3.9.3", "aiosignal==1.3.1", "albumentations==1.3.1", "altair==5.2.0", "antlr4-python3-runtime==4.9.3", "anyio==3.7.1", "async-timeout==4.0.3", "attrs==23.2.0", "av==11.0.0", "basicsr==1.4.2", "beautifulsoup4==4.12.3", "blendmodes==2022", "blis==0.7.11", "catalogue==2.0.10", "certifi==2024.2.2", "cffi==1.16.0", "chardet==5.2.0", "charset-normalizer==3.3.2", "clean-fid==0.1.35", "click==8.1.7", "clip-interrogator==0.6.0", "clip==1.0", "cloudpathlib==0.16.0", "colorama==0.4.6", "coloredlogs==15.0.1", "colorlog==6.8.2", "confection==0.1.4", "contourpy==1.2.0", "cssselect2==0.7.0", "cycler==0.12.1", "cymem==2.0.8", "cython==3.0.8", "deprecation==2.1.0", "depth-anything==2024.1.22.0", "diffusers==0.25.0", "dynamicprompts==0.23.0", "easydict==1.11", "einops==0.4.1", "embreex==2.17.7.post4", "exceptiongroup==1.2.0", "facexlib==0.3.0", "fastapi==0.94.0", "ffmpy==0.3.1", "filelock==3.13.1", "filterpy==1.4.5", "flatbuffers==23.5.26", "fonttools==4.48.1", "frozenlist==1.4.1", "fsspec==2024.2.0", "ftfy==6.1.3", "future==0.18.3", "fvcore==0.1.5.post20221221", "gdown==5.1.0", "gitdb==4.0.11", "gitpython==3.1.32", "gradio-client==0.5.0", "gradio==3.41.2", "grpcio==1.60.1", "h11==0.12.0", "handrefinerportable==2024.1.18.0", "httpcore==0.15.0", "httpx==0.24.1", "huggingface-hub==0.20.3", "humanfriendly==10.0", "idna==3.6", "imageio-ffmpeg==0.4.9", "imageio==2.33.1", "importlib-metadata==7.0.1", "importlib-resources==6.1.1", "inflection==0.5.1", "insightface==0.7.3", "iopath==0.1.9", "jinja2==3.1.3", "joblib==1.3.2", "jsonmerge==1.8.0", "jsonschema-specifications==2023.12.1", "jsonschema==4.21.1", "kiwisolver==1.4.5", "kornia==0.6.7", "langcodes==3.3.0", "lark==1.1.2", "lazy-loader==0.3", "lightning-utilities==0.10.1", "llvmlite==0.42.0", "lmdb==1.4.1", "lxml==5.1.0", "mapbox-earcut==1.0.1", "markdown-it-py==3.0.0", "markdown==3.5.2", "markupsafe==2.1.5", "matplotlib==3.8.2", "mdurl==0.1.2", "mediapipe==0.10.9", "mpmath==1.3.0", "multidict==6.0.5", "murmurhash==1.0.10", "networkx==3.2.1", "numba==0.59.0", "numexpr==2.9.0", "numpy==1.26.2", "omegaconf==2.2.3", "onnx==1.15.0", "onnxruntime==1.17.0", "open-clip-torch==2.20.0", "opencv-contrib-python==4.9.0.80", "opencv-python-headless==4.9.0.80", "opencv-python==4.9.0.80", "orjson==3.9.13", "packaging==23.2", "pandas==2.2.0", "piexif==1.1.3", "pillow==9.5.0", "pims==0.6.0", "pip==22.3.1", "platformdirs==4.2.0", "portalocker==2.8.2", "preshed==3.0.9", "prettytable==3.9.0", "protobuf==3.20.3", "psutil==5.9.5", "pycollada==0.8", "pycparser==2.21", "pydantic==1.10.14", "pydub==0.25.1", "pygments==2.17.2", "pyparsing==3.1.1", "pyreadline3==3.4.1", "pysocks==1.7.1", "python-dateutil==2.8.2", "python-multipart==0.0.9", "pytorch-lightning==1.9.4", "pytz==2024.1", "pywavelets==1.5.0", "pywin32==306", "pyyaml==6.0.1", "qudida==0.0.4", "referencing==0.33.0", "regex==2023.12.25", "reportlab==4.1.0", "requests==2.31.0", "resize-right==0.0.2", "rich==13.7.0", "rpds-py==0.17.1", "rtree==1.2.0", "safetensors==0.4.2", "scikit-image==0.21.0", "scikit-learn==1.4.0", "scipy==1.12.0", "semantic-version==2.10.0", "send2trash==1.8.0", "sentencepiece==0.1.99", "setuptools==65.5.0", "shapely==2.0.2", "six==1.16.0", "slicerator==1.1.0", "smart-open==6.4.0", "smmap==5.0.1", "sniffio==1.3.0", "sounddevice==0.4.6", "soupsieve==2.5", "spacy-legacy==3.0.12", "spacy-loggers==1.0.5", "spacy==3.7.2", "spandrel==0.1.6", "srsly==2.4.8", "starlette==0.26.1", "svg.path==6.3", "svglib==1.5.1", "sympy==1.12", "tabulate==0.9.0", "tb-nightly==2.16.0a20240210", "tensorboard-data-server==0.7.2", "termcolor==2.4.0", "tf-keras-nightly==2.16.0.dev2024021010", "thinc==8.2.3", "threadpoolctl==3.2.0", "tifffile==2024.1.30", "timm==0.9.12", "tinycss2==1.2.1", "tokenizers==0.13.3", "tomesd==0.1.3", "tomli==2.0.1", "toolz==0.12.1", "torch==2.1.2+cu121", "torchdiffeq==0.2.3", "torchmetrics==1.3.0.post0", "torchsde==0.2.6", "torchvision==0.16.2+cu121", "tqdm==4.66.1", "trampoline==0.1.2", "transformers==4.30.2", "trimesh==4.1.3", "typer==0.9.0", "typing-extensions==4.9.0", "tzdata==2023.4", "urllib3==2.2.0", "uvicorn==0.27.1", "vhacdx==0.0.5", "wasabi==1.1.2", "wcwidth==0.2.13", "weasel==0.3.4", "webencodings==0.5.1", "websockets==11.0.3", "werkzeug==3.0.1", "xformers==0.0.23.post1", "xxhash==3.4.1", "yacs==0.1.8", "yapf==0.40.2", "yarl==1.9.4", "zipp==3.17.0" ] }

Console logs

** Error running process_before_every_sampling: C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\Users\dervl\stable-diffusion-webui-forge\modules\scripts.py", line 830, in process_before_every_sampling
        script.process_before_every_sampling(p, *script_args, **kwargs)
      File "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 533, in process_before_every_sampling
        self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, **kwargs)
      File "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 479, in process_unit_before_every_sampling
        params.model.process_before_every_sampling(p, cond, mask, *args, **kwargs)
      File "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_ipadapter\scripts\forge_ipadapter.py", line 147, in process_before_every_sampling
        unet = opIPAdapterApply(
      File "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 724, in apply_ipadapter
        self.ipadapter = IPAdapter(
      File "C:\Users\dervl\stable-diffusion-webui-forge\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 287, in __init__
        self.image_proj_model.load_state_dict(ipadapter_model["image_proj"])
      File "C:\Users\dervl\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
        raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
    RuntimeError: Error(s) in loading state_dict for ImageProjModel:
        size mismatch for proj.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([3072, 1280]).

Additional information

No response

[lllyasviel]

it works for me without problems. maybe wrong model version?

[Saranmt]

I can't see any of the other ip adapter preprocessors, updated but still only shows these 3

[nharwell]

I have the same issue - I'm no longer seeing any of the ip-adapter preprocessors except those 3.

[aleph23]

3rd confirm: I just git-cloned forge on top of an updated auto1111 v1.7.0-dev-branch. Here's my full list of preprocessors -- with just three IPadapters.

[lhucklen]

I do not have the preprocessors anymore; how can I resolve this?

[XiangjiBU]

I can't see any of the other ip adapter preprocessors, updated but still only shows these 3

Use the third "CLIP-ViT-H (IPAdapter)" not "InsightFace+CLIP-H (IPAdapter)". [https://github.com/tencent-ailab/IP-Adapter] use "CLIP-VIT-H"

by the way, I met problem too when using "InsightFace+CLIP-H (IPAdapter)" and "CLIP-ViT-bigG (IPAdapter)", only "CLIP-ViT-H (IPAdapter)" works well

[albertocasanova]

4th confirm, missing the same preprocessors.

[CRCODE22]

There is no preprocessors for xl models in forge but there is in automatic 1111

[CRCODE22]

I copied and pasted all the files needed in the \stable-diffusion-webui-forge\models\ControlNet yet forge fails to see the xl preprocessors. All the xl models do show up but no preprocessors for xl.

[albertocasanova]

There's missing preprocessors for both SDXL and SD, as the post title indicates.

[lllyasviel]

Hi please note that the previous naming of ipadapters preprocessors in sd-webui-controlnet were wrong. The names in Forge are correct and consistent to official naming from ipadapter official :

SD v.	IPadapter	Img encoder
v1.5	ip-adapter_sd15	ViT-H
v1.5	ip-adapter_sd15_light	ViT-H
v1.5	ip-adapter-plus_sd15	ViT-H
v1.5	ip-adapter-plus-face_sd15	ViT-H
v1.5	ip-adapter-full-face_sd15	ViT-H
v1.5	ip-adapter_sd15_vit-G	ViT-bigG
SDXL	ip-adapter_sdxl	ViT-bigG
SDXL	ip-adapter_sdxl_vit-h	ViT-H
SDXL	ip-adapter-plus_sdxl_vit-h	ViT-H
SDXL	ip-adapter-plus-face_sdxl_vit-h	ViT-H
v1.5	FaceID	InsightFace+CLIP-H
v1.5	FaceID Plus	InsightFace+CLIP-H
v1.5	FaceID Plus v2	InsightFace+CLIP-H
v1.5	FaceID Portrait	InsightFace+CLIP-H
SDXL	FaceID	InsightFace+CLIP-H
SDXL	FaceID Plus v2	ViT-H

[CRCODE22]

Also why does this build in controlnet not have the option for multiple input images? The not build in version that you can install through extensions does have it but you cannot use that in forge because forge keeps disabling it even when checked. In automatic 1111 it works and has multiple input images.

[CRCODE22]

Do I really need to make a new issue for this multiple input images option missing and the main branch of controlnet being disabled by forge even tough I mark it as enabled? The build in controlnet is missing a very important feature.

[CRCODE22]

This controlnet in automatic 1111 has multi input image option I am going back to automatic 1111 forge is not good at all. If does not allow me to use a controlnet with the below multi input image option which you can use in automatic 1111.

[lllyasviel]

@CRCODE22

Hi please note that the multi input image in previous sd-webui-controlnet was wrong and misleading

the previous multi input image was equivalent to uploading images to multiple units with same settings

sd-webui-controlnet never had a real multi input controlnet

the multi-upload in Forge is under construction and will be used by animatediff in a correct way.

[FruitPigFoot]

@lllyasviel I've tested both webui and webui-forge, running same options with same models. I used controlnet inpaint, canny and 3 ip-adapter units each with one style image. However, the results seems quite different. I also tried ip-adapter image with original sizes and also cropped to 512 size but it didn't make any difference. Below is the result this is the result image with webui's controlnet this is the result image with webui-forge's integrated controlnet.

results vary a little after each generation. However, most of the times, webui result seems more natural. Are some loading methods or generation method different enough to cause these difference?

[strawberrymelonpanda]

the multi-upload in Forge is under construction and will be used by animatediff in a correct way.

@lllyasviel How about IP-Adapter, will it be able to use the new multi-upload as well?

Personally I did not consider the previous behavior to be wrong, although a quick glance through Comfy's "image batch" function (not the same "batch" as batch counts) shows it's basically just doing torch.cat on image arrays. There are workflows where it is very useful.

image batch

def batch(self, image1, image2):
  if image1.shape[1:] != image2.shape[1:]:
    image2 = comfy.utils.common_upscale(image2.movedim(-1,1), image1.shape[2], image1.shape[1], "bilinear", "center").movedim(1,-1)
  s = torch.cat((image1, image2), dim=0)
  return (s,)

[K-Z007]

Came across this, I got the same issue, I've done a number of comparisons/tests with correct processor in webui version it really gives me a very similarly consistent faces. However without selecting correct processor with ip-adater-plus in webui forge version, it generates faces are slightly different. The 3 default processors worked but not give me the correct results which would be able to achieve in webui. Hope we can get a fix, thank you a lot.

[chandhuexe]

I copied and pasted all the files needed in the \stable-diffusion-webui-forge\models\ControlNet yet forge fails to see the xl preprocessors. All the xl models do show up but no preprocessors for xl.

you got any solution?

[jacquesfeng123]

2024-05-07 17:33:54,817 - ControlNet - INFO - ControlNet Input Mode: InputMode.SIMPLE 2024-05-07 17:33:54,853 - ControlNet - INFO - Using preprocessor: CLIP-ViT-H (IPAdapter) 2024-05-07 17:33:54,854 - ControlNet - INFO - preprocessor resolution = 832 2024-05-07 17:33:54,861 - ControlNet - INFO - Current ControlNet IPAdapterPatcher: /home/lam/stable-diffusion-webui/models/ControlNet/ip-adapter-plus_sdxl_vit-h.safetensors INFO: the IPAdapter reference image is not a square, CLIPImageProcessor will resize and crop it at the center. If the main focus of the picture is not in the middle the result might not be what you are expecting. *** Error running process_before_every_sampling: /home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/scripts/controlnet.py Traceback (most recent call last): File "/home/lam/stable-diffusion-webui-forge/modules/scripts.py", line 835, in process_before_every_sampling script.process_before_every_sampling(p, *script_args, **kwargs) File "/home/lam/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/scripts/controlnet.py", line 555, in process_before_every_sampling self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, **kwargs) File "/home/lam/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/scripts/controlnet.py", line 501, in process_unit_before_every_sampling params.model.process_before_every_sampling(p, cond, mask, *args, **kwargs) File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_ipadapter/scripts/forge_ipadapter.py", line 147, in process_before_every_sampling unet = opIPAdapterApply( File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_ipadapter/lib_ipadapter/IPAdapterPlus.py", line 723, in apply_ipadapter self.ipadapter = IPAdapter( File "/home/lam/stable-diffusion-webui-forge/extensions-builtin/sd_forge_ipadapter/lib_ipadapter/IPAdapterPlus.py", line 286, in init self.image_proj_model.load_state_dict(ipadapter_model["image_proj"]) File "/home/lam/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([1280, 1664]).

hi, we got carried away. I am facing this issue too.

I should be using the correct preprocessor and model.

model redownloaed from the link the author gave, and preprocessor CLIP-ViT-H (IPAdapter), I am using SDXL endpoint