[enhancement]: Add an extensions or addons tab

[ParisNeo]

Is there an existing issue for this?

• [X] I have searched the existing issues

Contact Details

No response

What should this feature add?

Is it possible to add a tab for external extensions? This will allow people to build and add new extensions to the ui by adding new scripts that can do pretty cool stuff. This is already supported by the AUTOMATIC1111 / stable-diffusion-webui repository.

On that repository therer are really cool extensions that really change the game But It should be a good idea to have an equivalent tool and even better, a compatible tool as invokeai is more user friendly and beautiful ui.

Does anyone think that this could be a really cool feature to have?

Alternatives

No response

Aditional Content

No response

[dhelonious]

I am not sure if I like this idea. This will lead to a lot of broken and unmaintained scripts, as it is the case in auto1111. For me this contradicts the concept of invokeai to provide a stable set of tools for ai art. Further, I do not see any issues in using both auto1111 and invokeai together. Game changing things can always be brought to invokeai via pull requests with reviews and quality control.

[ParisNeo]

We can authorize only scripts that have been validated for the current version. In fact, not all people for example need a tool that can use a pose or a depth map to generate an image. If you integrate this into the main tool, it would make the ui very crouded and will make the tool heavy and ugly. Extensions enable people to load stuff they want when and if they need it.

I am teaching the use of those tools to people who are not developers and having to teach many tools is a little unsettling. auto1111 is more complete with loads of cool stuff. But I prefer largely invokeai in terms of quality and it seams to me that options should be optional and many famous tools have embraced this kind of extensions :

vscode jupyterlab Google chrome, opera, firefox, edge ...

to mension just few.

[dhelonious]

If extension support is added to invoke, this should be done properly by providing APIs, tools, and security checks for good integrations. In my understanding this is far from trivial and requires a lot of work. And regarding the extensible tools you mentioned, I don't think the devs at Google, Mircrosoft, and Mozilla are developing them in their spare time ;-)

Invoke has such an amount of quality because the devs focus on stability and polishment of a specific set of features rather than on implementing the newest stuff quick and dirty. And in my opinion the focus should remain on the core functionality and unique features like the upcoming node system. Extension support should only then be added when the base app is mature enough and the API is stable. Otherwise bigger updates would probably break all extensions (which already happened in auto1111) or, to prevent this, the devs have to be extra careful which hinders the development. Though, these are just my personal opinions and concerns. Ultimately an extension support would surely be nice to have.

[ParisNeo]

I definitely agree with you that we shouldn't fall into the feature creep situation. This project should focus on the nodes feature which I am really looking forward to. I also agree that if an extension system is built it should be done via an API that should garentee interoperability between versions.

I think it is a nice to have feature as you mensioned. Maybe not a priority for now, but keep an eye on it.

It is just that as a user and teacher, I find it important to stick to one clear tool and invokeai seems to be the right one as it is really well built. The ui is intuitive and way better than the auto1111 ui. But when I want to teach more advanced stuff, I need to use auto1111 as there is no equivalent in invokeai and things start to get technical. Even textual inversion is easyer and more configurable using the auto1111 (in invoke ai it is still only supported through the console) and there is much stuff that needs to be added like using BLIP for captionning images etc. There is no Lora no hypernetworks not even dreambooth as far as I know. People may say Textual inversion is the best, but it depends on the origial network as you are searching an embedding which means that if the original network have never seen something like what you are teaching it, it is complicated to get good results.

I have introduced this course to the architecture school this year as an experiment because I think this could be a game changer. They can visulize concepts and hand craft new ideas and some extensions are just right for architects and can save a huge amount of time. So this may definetely become a must learn thing for architects. It is an experiment this year that may become a fixed course in the future if i my course manages to convince the board.

Making a course for one tool is easyer than multiple tools. :). I also have limited time for this course so I have to present as much as I can in a short time.

But nevermind. I will be teaching both. I hope this won't be complicated for them. I also want to contribute to this project. For now I just did dome localization work (french) but maybe I can participate in more AI related work as this is my area of speciality.

Thanks for replying.

[dhelonious]

I completely agree that having one good tool with a well-designed UI would be great. I have no doubt that invokeai will one day be able to fulfill this role, as every update is just impressive and brings lots of features with it. I am also looking forward to the GUI for textual inversion, as I am also still using auto1111 for training at the moment. I read that the devs are working on LORA support (and I hope to see also LORA training). Dreambooth would be nice, but most users can probably not use it locally (like myself).

I had some second thoughts about your suggestion for an extension tab. The node system could be a good place to add extension support for custom nodes. This would prevent cluttering the UI with extensions and would not require a full extension API.

Your project to bring AI into architecture sounds great! I agree that AI as a tool could really help architects with their projects and make their workflows more efficient. Hopefully your endeavour is spared by the "AI is not art" debate, since there are lots of architects which see themselves as artists rather than engineers (at least in my experience, I studied architecture for some semesters). But AI is a game changer for our society as a whole, in the long run it will probably be impossible to not employ the tools somehow. Best of luck with your project! I hope you will share this as a user story in the future :)

[ebr]

Ability to extend InvokeAI is the next priority on the roadmap, and is already being worked in #1650. We are building a robust API that will allow users to write their own plugins in a more structured fashion than is currently available with other Web UIs.

This is a great topic though, thank you both for your perspectives. If you'd like, we could convert the issue to a discussion, or continue the conversation in Discord: https://discord.com/channels/1020123559063990373/1049107548264992779.