adetailer icon indicating copy to clipboard operation
adetailer copied to clipboard
verified publisher icon Bing-su

[Bug]: When Using [SEP] in Adetailer Prompt, only 1 face's prompt is added to the image metadata

Open FriedGenera opened this issue 1 year ago • 4 comments

Describe the bug

As the title suggests, when you use [SEP] in adetailer's prompt, only one of the face's prompts is saved to the image's metadata. This applies to any generation with more than 1 face, all other faces are not added to the metadata.

Steps to reproduce

Enabling Adetailer, add prompt with [SEP], enable regional prompter, generate the image with Hi-Res fix and controlnet models.

Screenshots

edit

Console logs, from start to end.

Python 3.10.6 | packaged by conda-forge | (main, Oct 24 2022, 16:02:16) [MSC v.1916 64 bit (AMD64)]
Version: v1.9.4
Commit hash: feee37d75f1b168768014e4634dcb156ee649c05
Installing sd-webui-controlnet requirement: changing opencv-python version from 4.10.0.84 to 4.8.0
Launching Web UI with arguments: --theme dark --xformers
[-] ADetailer initialized. version: 24.6.0, num models: 14
Using sqlite file: ~\extensions\sd-webui-agent-scheduler\task_scheduler.sqlite3
ControlNet preprocessor location: ~\extensions\sd-webui-controlnet\annotator\downloads
2024-07-07 20:38:54,728 - ControlNet - INFO - ControlNet v1.1.452
~\diffusers\models\transformers\transformer_2d.py:34: FutureWarning: `Transformer2DModelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
  deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
Loading weights [54ef3e3610] from ~\models\Stable-diffusion\meinamix_meinaV11.safetensors
2024-07-07 20:38:55,864 - ControlNet - INFO - ControlNet UI callback registered.
Creating model from config: ~\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 19.0s (prepare environment: 7.4s, import torch: 3.8s, import gradio: 0.8s, setup paths: 1.0s, initialize shared: 0.2s, other imports: 0.6s, load scripts: 2.4s, create ui: 1.5s, gradio launch: 0.2s, app_started_callback: 1.1s).
Applying attention optimization: xformers... done.
Model loaded in 3.4s (load weights from disk: 0.5s, create model: 0.5s, apply weights to model: 2.1s, calculate empty prompt: 0.1s).
2024-07-07 20:55:56,206 - ControlNet - INFO - unit_separate = False, style_align = False
2024-07-07 20:55:56,432 - ControlNet - INFO - Loading model: control_v11p_sd15_openpose [cab727d4]
2024-07-07 20:55:58,673 - ControlNet - INFO - Loaded state_dict from [~\extensions\sd-webui-controlnet\models\control_v11p_sd15_openpose.pth]
2024-07-07 20:55:58,674 - ControlNet - INFO - controlnet_default_config
2024-07-07 20:56:00,550 - ControlNet - INFO - ControlNet model control_v11p_sd15_openpose [cab727d4](ControlModelType.ControlNet) loaded.
2024-07-07 20:56:00,569 - ControlNet - INFO - Using preprocessor: none
2024-07-07 20:56:00,569 - ControlNet - INFO - preprocessor resolution = 512
2024-07-07 20:56:00,576 - ControlNet - INFO - Loading model: control_v11f1p_sd15_depth [cfd03158]
2024-07-07 20:56:01,159 - ControlNet - INFO - Loaded state_dict from [~\extensions\sd-webui-controlnet\models\control_v11f1p_sd15_depth.pth]
2024-07-07 20:56:01,159 - ControlNet - INFO - controlnet_default_config
2024-07-07 20:56:02,857 - ControlNet - INFO - ControlNet model control_v11f1p_sd15_depth [cfd03158](ControlModelType.ControlNet) loaded.
2024-07-07 20:56:02,864 - ControlNet - INFO - Using preprocessor: none
2024-07-07 20:56:02,864 - ControlNet - INFO - preprocessor resolution = 512
2024-07-07 20:56:03,092 - ControlNet - INFO - Loading model: control_v11p_sd15_lineart [43d4be0d]
2024-07-07 20:56:03,642 - ControlNet - INFO - Loaded state_dict from [~\extensions\sd-webui-controlnet\models\control_v11p_sd15_lineart.pth]
2024-07-07 20:56:03,642 - ControlNet - INFO - controlnet_default_config
2024-07-07 20:56:05,360 - ControlNet - INFO - ControlNet model control_v11p_sd15_lineart [43d4be0d](ControlModelType.ControlNet) loaded.
2024-07-07 20:56:05,368 - ControlNet - INFO - Using preprocessor: none
2024-07-07 20:56:05,368 - ControlNet - INFO - preprocessor resolution = 512
2024-07-07 20:56:05,401 - ControlNet - INFO - ControlNet Hooked - Time = 9.201756477355957
1,1 0.2 Horizontal
Regional Prompter Active, Pos tokens : [49, 52], Neg tokens : [15]
100%|██████████████████████████████████████████████████████████████████████████████████| 60/60 [00:38<00:00,  1.55it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [00:37<00:00,  1.82it/s]
0: 320x640 2 faces, 95.0ms
Speed: 4.7ms preprocess, 95.0ms inference, 47.3ms postprocess per image at shape (1, 3, 320, 640)
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:03<00:00,  8.24it/s]
[-] ADetailer: applied 1st ad_prompt: '<lora:ubel_v1:0.9>, ubel, grinning'
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:02<00:00, 10.98it/s]
[-] ADetailer: applied 1st ad_prompt: 'scowl, angry, aafern, <lora:fern_(sousou_no_frieren)_v1:0.8>'

List of installed extensions

Adetailer Controlnet Regional Prompter Agent Scheduler

FriedGenera avatar Jul 08 '24 04:07 FriedGenera

Hello, @FriedGenera! I'm here to help you with any bugs, questions, or contributions. Let me know how I can assist you.

Using [SEP] in Adetailer's prompt results in only one face's prompt being saved to the image's metadata because the save_image function only saves a single prompt from p.all_prompts or p.prompt to the image's metadata. The function does not handle multiple prompts separated by [SEP].

Here is the relevant code snippet from the save_image function:

def save_image(self, p, image, *, condition: str, suffix: str) -> None:
    i = get_i(p)
    if p.all_prompts:
        i %= len(p.all_prompts)
        save_prompt = p.all_prompts[i]
    else:
        save_prompt = p.prompt
    seed, _ = self.get_seed(p)

    if opts.data.get(condition, False):
        images.save_image(
            image=image,
            path=p.outpath_samples,
            basename="",
            seed=seed,
            prompt=save_prompt,
            extension=opts.samples_format,
            info=self.infotext(p),
            p=p,
            suffix=suffix,
        )

The save_prompt variable is set to a single prompt from p.all_prompts or p.prompt, and this single prompt is saved to the image's metadata [1].

To continue talking to Dosu, mention @dosu.

dosubot[bot] avatar Jul 08 '24 04:07 dosubot[bot]

I previously applied a feature to show the actual applied prompt in the AD Prompt field. I think this is causing this bug now and would like to revert it.

Thanks.

Bing-su avatar Jul 12 '24 05:07 Bing-su

I am experiencing this issue as well. Same steps to reproduce. It only applies one of the prompts separated by [SEP] even though it detects more than one face in the image.

Tarho-sd avatar Jul 31 '24 22:07 Tarho-sd

I probably encountered the same situation.

After upgrading from ADetailer:v24.1.2 to v24.6.0, the prompts in the prompt column, [SEP] and the prompts after it, are no longer recorded in png. It seems that [SEP] and the prompts after it are disabled.

Since they are not recorded in png, I cannot reproduce the previous settings.

For the time being, I have reverted to v24.1.2 to deal with the problem.

Environment mediapipe_face_full A1111 webUI:v1.7.0 M2 mac mini

elistys avatar Aug 02 '24 23:08 elistys