InvokeAI
InvokeAI copied to clipboard
invoke-ai
[enhancement]: ControlNet is needed here
Is there an existing issue for this?
- [X] I have searched the existing issues
Contact Details
No response
What should this feature add?
ControlNet is needed here, it is a very very powerful feature for stable diffusion. Now, SD webui gets it.
It's an img2img feature, it takes the source image and use ControlNet AI to generate this image's frame lines or character's pose. So the output image can get the samething but keeps its own style
check it out: https://github.com/lllyasviel/ControlNet
and this is SD webui's extension for this: https://github.com/Mikubill/sd-webui-controlnet
yes, please. i love invokeai's ui and everything but controlnet is too awesome to miss.
Anyone working on this? Or anyone could at least lay out the steps that need to be taken for this to work?
Yeah, I'm already thinking of installing the old SD again just to use it to speed up some projects. It's very much needed for Invoke.
The real deal is, invokeAI need to support extensions. So other developer can add new feature to InvokeAI.
For example, I created a SD webui's extension to translate native language into English, so now, people can use their native language as prompt.
I'd like to add this feature to InvokeAI, but it seems does not support extensions.
The real deal is, invokeAI need to support extensions. So other developer can add new feature to InvokeAI.
See also this newly-created issue: https://github.com/invoke-ai/InvokeAI/issues/2696
Although one could also just fork the repo, implement ControlNet support, and create a pull request?
There's active work on getting this in to the upstream diffusers library:
- https://github.com/huggingface/diffusers/issues/2331
any interface InvokeAI adds for ControlNet is likely to be built on top of that.
Although one could also just fork the repo, implement ControlNet support, and create a pull request?
Yes, PRs always welcome. The #developer-forums or #dev-chat on discord is the best place to coordinate that sort of thing. (See #739 for access to #dev-chat.)
Hi guys, I opened a ControlNet PR on Diffusers repo yesterday. Still in WIP. If you have any comments from a user perspective, especially on Discussion-1, please let me know.
The real deal is, invokeAI need to support extensions. So other developer can add new feature to InvokeAI.
Yes the lack of extensions support I think really hamstrings InvokeAI and limits the speed at which features can be added. It would be great to see this implemented.
Are extensions even planned, i mean it seems like thats the 1 thing holding back invoke from being the defacto standard vs A111, most people jump on that one because due to extensions its the first to get new things... Seems some form of extension pipeline should be part of the InvokeAI even if those better extensions eventually get rolled into main system.
Going to chime in agreeing with the extension concept, if that will help get the features over to InvokeAI sooner. Automatic111 crashes for me (and freezes my computer) regularly, InvokeAI has not done that even once. So there is a more stable app here, and perhaps that's in part because it didn't go the route of extensibility, but I think finding a way to do it would be great.
I'm more moderate; I think that this topic (controlNET) is needed as a feature, but I think Invoke has a leg up in:
- UI
- Stability
- Ease of use
Extensibility IMO should take a backseat vs retaining the above for end users (in terms of actively supporting/developing ways to do this), however it probably wouldn't hurt to at least make some extension documentation to get people started.
I'm more moderate; I think that this topic (controlNET) is needed as a feature, but I think Invoke has a leg up in:
* UI * Stability * Ease of useExtensibility IMO should take a backseat vs retaining the above for end users (in terms of actively supporting/developing ways to do this), however it probably wouldn't hurt to at least make some extension documentation to get people started.
I agree with this. This is how Invoke AI stands out. Although I appreciate the developers of the automatic1111 Web UI, there is bad UI and no UX there. Whatever you decide to do, you have to either hack your way through or find someone's guide who had already hacked their way through the task.
I use Automatic1111 for testing out the bleeding edge features. For actually using SD for a real-world task, Invoke AI is the go-to app.
Are extensions even planned, i mean it seems like thats the 1 thing holding back invoke from being the defacto standard vs A111, most people jump on that one because due to extensions its the first to get new things... Seems some form of extension pipeline should be part of the InvokeAI even if those better extensions eventually get rolled into main system.
They are working on it.
There are two major issues with Invoke that put it behind A1111 right now and this (and the lack of extension support in general) is one of them. Extension support in general is badly needed.
With Diffusers 0.14.0, ControlNet is included in diffusers by default.
So it should be easy-ish to include it? I don't know how the internals work, but yeah. It would probably require an update to the web ui as well. Or maybe just load the various checkpoints into the model dropdown.
So there's some design that needs to happen, but still.
love Invoke AI, looking forward to the ControlNet features supported / included. Thank you guys~
Controlnet 1.1 has been released, and it is significantly better than the previous version, cnet 1.0. I am eagerly wait for the controlnet and batch img2img generation features to become available so that I can switch from using the A1111 webui to using invokeai.
I'm very much looking forward to this being available in InvokeAI. I'm new here, but it's been fantastic to use, and this would add another level of control to form generations on top of img2img.
