How do u get this to run in automatic1111?

[ice051128]

Any guide?

[78Alpha]

Aside from replacing the core parts of the UI and just dropping this in (breaking everything else in the WebUI as a result), it would be a lot of manual code porting.

• Support TRT models for import
• Adapt Samplers to TRT
• Add model conversion to the UI or some documentation to an external tool
• Attempt to integrate optimizations (Split attention, Xformers, other things) to see if they even work with it
• If it has no native windows version (requires WSL) then it may just get dropped like a rock due to the massive amount of support issues that would pop up

[ice051128]

Aside from replacing the core parts of the UI and just dropping this in (breaking everything else in the WebUI as a result), it would be a lot of manual code porting.

• Support TRT models for import
• Adapt Samplers to TRT
• Add model conversion to the UI or some documentation to an external tool
• Attempt to integrate optimizations (Split attention, Xformers, other things) to see if they even work with it
• If it has no native windows version (requires WSL) then it may just get dropped like a rock due to the massive amount of support issues that would pop up

It's unfortunate to see that this has no real uses for stable diffusion

[VoltaML]

Aside from replacing the core parts of the UI and just dropping this in (breaking everything else in the WebUI as a result), it would be a lot of manual code porting.

• Support TRT models for import
• Adapt Samplers to TRT
• Add model conversion to the UI or some documentation to an external tool
• Attempt to integrate optimizations (Split attention, Xformers, other things) to see if they even work with it
• If it has no native windows version (requires WSL) then it may just get dropped like a rock due to the massive amount of support issues that would pop up

It's unfortunate to see that this has no real uses for stable diffusion

Can you please elaborate? A lot of users have found our version very fast and reaping the benefits for fast inference. But we are still trying our best to find integration support with automatic.

[78Alpha]

I assume it has to do with the fact Automatic1111 has become the unofficial face of stable diffusion. Things that aren't integrated there tend to get forgotten or dropped. A few groups were working on tensor RT for automatic a few months ago but integration slowed and eventually stopped.

Another part may be the VRAM requirement. Most are used to being able to run on 4 GB to 8 GB VRAM. A model that is faster but raises that entry level bar instead of lowering it might not sit will.

[harishprabhala]

I assume it has to do with the fact Automatic1111 has become the unofficial face of stable diffusion. Things that aren't integrated there tend to get forgotten or dropped. A few groups were working on tensor RT for automatic a few months ago but integration slowed and eventually stopped.

Another part may be the VRAM requirement. Most are used to being able to run on 4 GB to 8 GB VRAM. A model that is faster but raises that entry level bar instead of lowering it might not sit will.

Well it's unfortunate that people are not going other projects a chance. Integrate into automatic or die is not a good way to support innovation. Hope more users start encouraging other projects as well.

[jrd-rocks]

I assume it has to do with the fact Automatic1111 has become the unofficial face of stable diffusion. Things that aren't integrated there tend to get forgotten or dropped. A few groups were working on tensor RT for automatic a few months ago but integration slowed and eventually stopped. Another part may be the VRAM requirement. Most are used to being able to run on 4 GB to 8 GB VRAM. A model that is faster but raises that entry level bar instead of lowering it might not sit will.

Well it's unfortunate that people are not going other projects a chance. Integrate into automatic or die is not a good way to support innovation. Hope more users start encouraging other projects as well.

My 2cts, for what they're worth. The lowest level libraries currently are diffusers and stablediffusion, which uses pytorch-lightning. Since this project is more about foundational work than ui, i'd not worry too much about ui or integrations in ui's, just integrate well with the low level libraries, then third party ui's can build further on that. Just document/create an api on how to do that.

Having said that, it might be challenging enough, since to my limited understanding you build a new execution graph for the entire model, while it's common to plug in at certain layers in the model, think hypernetworks or skipping the last clip layers for guidance, thus creating a new execution graph.

[78Alpha]

May be unrelated in a direct means, but it is starting to seem like the windows option is out the... window. TensorRT is not an easy installation process. Only gives option of 0.0.1-dev and attempting to download the files to build from nvidia sent me to a page that wouldn't allow me to login (this is probably a personal account problem).

That does break the Automatic1111 design of "launch the bat and it will setup everything".

I would have to second going the dep down route of the diffusers. In the long run it should make everything easier (hopefully).

Adding in an edit, did end of getting tensor RT to install, had to setup a second environment, sadly VoltaML runs and then... Doesn't. No OOM or anything, just eventually stops and goes back to command prompt.

[wywywywy]

It saddens me to say this, but it's probably a better idea to integrate with InvokeAI than A1111.

InvokeAI has a much better project structure, a proper licence, and is very actively maintained. It also already has a functional Docker setup.

With A1111, there's only one main person. As soon as A1111 loses interest (which might have already happened), it'll die. InvokeAI has a much brighter future.