sd-webui-controlnet
sd-webui-controlnet copied to clipboard
Mikubill
[Bug]: Newer versions break prompt travel script (aaaand it's back :Þ )
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui
What happened?
ControlNet used to work alongside prompt travel, both in txt2img and img2img; I used it regularly (I can share videos). After not having updated for a couple weeks, updated controlnet - loving the new features, but not loving that prompt travel no longer works. :( The first two frames work, but then it goes to chaos.
All further frames are like the last frame above.
Steps to reproduce the problem
Prompt: Heart. High quality. HDR. 8K. High definition. Award-winning. Reuters. HQ. Cow. Heart. High quality. HDR. 8K. High definition. Award-winning. Reuters. HQ. Sampler: DPM++ 2M Karras Steps: 6 Width: 1920 Height: 1088 CFG: 7 Seed: 0 Control Model 0: Disabled Control Model 1: Image: <See Below> Weight: 0.75 Guidance Start: 0 Guidance End: 0.25 Resize mode: Just Resize Canvas width: 1920 Canvas height: 1088 Script: Prompt travel: All default, except: Travel between stages: 40
What should have happened?
It should have continued along with prompt travel, like used to work, instead of going to chaos.
Commit where the problem happens
webui:\t\t controlnet:\t
Some time in the last couple weeks. I suspect it was the addition of Guidance Start/End - my totally-unsupported random guess is that when you click generate, it starts counting steps to decide when to enable/disable a network, but doesn't reset it with each new image generated. That guess could be entirely wrong, though!
What browsers do you use to access the UI ?
Google Chrome
Command Line Arguments
Normal args
Console logs
Nothing special.
Additional information
In case you're not familiar with prompt-travel (it's a commonly used extension, part of the standard list of AUTOMATIC1111 extensions):
https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel
After not having updated for a couple weeks
you could check your controlnet extension's previous commit versions you used, with git reflog. just right-click in the extensions/sd-webui-controlnet folder and select open in terminal.
Also, you can switch back to that version with git checkout YOURHASH
I know I can roll back, but obviously this is a bug that should be fixed, and hence, I reported it :)
Seems to have happened between cc1abdf on 15 Feb and 0c45b5a on 26 Feb. Will see if I can narrow it down to a specific commit.
Okay, I narrowed it down:
Last good commit: dc3309d First bad commit: 559395f
Thus the problem would appear to be:
commit 559395f4e0ff0ea0ed2bd3f9b989e2c7a1dfdd59 (HEAD -> main) Author: Sang [email protected] Date: Tue Feb 21 20:54:49 2023 -0800
fixed api
Closing as no reply from OP. I assume it works now? Feel free to reopen if it still does not work.
Just updated yet again, and... it's back. :( Some new commit seems to have broken it again. Will try to see if I can figure out which one...
Okay, tracked it down. Last working commit:
317723e - Fri Mar 10 17:38:55 2023 +0900
First broken commit:
f39578a - Fri Mar 10 16:42:41 2023 +0700
Changes:
--- a/scripts/controlnet.py
+++ b/scripts/controlnet.py
@@ -825,11 +825,11 @@ class Script(scripts.Script):
save_image(img, detectmap_dir, module)
is_img2img_batch_tab = is_ui and is_img2img and img2img_tab_tracker.submit_img2img_tab == 'img2img_batch_tab'
- no_detectmap_opt = shared.opts.data.get("control_net_no_detectmap", False)
- if self.latest_network is None or no_detectmap_opt or is_img2img_batch_tab:
+ if self.latest_network is None or is_img2img_batch_tab:
return
- if hasattr(self, "detected_map") and self.detected_map is not None:
+ no_detectmap_opt = shared.opts.data.get("control_net_no_detectmap", False)
+ if not no_detectmap_opt and hasattr(self, "detected_map") and self.detected_map is not None:
for detect_map, module in self.detected_map:
if detect_map is None:
continue
And apparently the commit author who broke it is refusing to take responsibility for their actions. :þ
https://github.com/Mikubill/sd-webui-controlnet/commit/f39578a9b0b41c9a41687ed9f8a27a952d176ece#commitcomment-105693452
No, the commit author is still refusing to take responsibility for their actions, and thinks it's perfectly fine for them to break scripts rather than actually make sure execution is complete before unload, to unload while AUTOMATIC1111's buttons still says "Interrupt" / "Skip" rather than "Generate" and it's in the middle of a run() function. Perfectly fine coding, from their point of view.
We're yet to hear @Mikubill's take.
what a pity, you said that there is some way to use it maybe with an old version of this extension, how to do it?
Yes. Go into the controlnet extension directory and run:
git checkout 317723e
or..
git reset --hard 317723e
(The latter will throw out any local changes if you've made any)
That'll bring you back to the last working version, from 10 March.
(And yes, prompt travel with ControlNet is awesome :) )
I am not completely sure I grasp the issue. IIUC, as this is related to running controlnet from external code, maybe one way to fix this could be to release resources when control units are from the ui, and resort to the previous behavior when called from a script. Could that work in your opinion @boringcrypto?
It has nothing to do with external code itself, but rather is core at how AUTOMATIC1111's scripting mechanism works as a whole. As noted in the other thread:
- UI initiates generation, then calls..
- txt2img(), which calls
- modules.scripts.scripts_txt2img.run (with the txt2img processing arguments provided), so that run() is called in the scripts (which in the case of scripts like prompt travel will make a number of calls, that ultimately lead to..
- multiple calls to process_images(), which calls: -- process() -- before_process_batch()- -- process_batch() -- postprocess_batch() --postprocess_image() -- postprocess()
- Repeat creating images as much as is needed.
- run() then ends, returning back to txt2img.
- txt2img() then ends, returning back to wherever in the UI called it.
- The UI then stops generation and changes the buttons back.
This isn't about the code of some external script - it's about the code of AUTOMATIC1111 itself. Just giving up at the first postprocess(), as boringcrypto did, is inherently going to break all scripts that generate more than one image. It should go without saying, but breaking AUTOMATIC1111's scripting mechanism isn't okay.
If you're going to release resources at postprocess() then you need to restore them at process(). That said, I don't really care what needs to be done to fix it. if it came down to even something as ugly as a settings checkbox to never release resources, even that would be greatly superior to the current situation, as then at least people would have a workaround for this faulty scripting-breaking behavior.
As an example of what you can do with this, when it actually is allowed to work:
https://youtu.be/zUa0pB_TqqM
It'll be many times more powerful once you can combine it with batch masks!
(The procedure used to create the above could also due to be refined further)
It seems this has been fixed on prompt travel side? @enn-nafnlaus have you seen the discussion at https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel/issues/18 ?
It seems this has been fixed on prompt travel side? @enn-nafnlaus have you seen the discussion at Kahsolt/stable-diffusion-webui-prompt-travel#18 ?
Interesting! No, I hadn't seen that, but as soon as my GPU is free, I'll try it out!
Awesome! I'll close this now, feel free to come back here or open a new issue if it breaks in the future 👍
Prompt travel + Controlnet fail again, generating noise for any steps between the start and end prompts.
