very interesting extension, can I include some?

[picobyte]

Directed from here, it seems you have an extension that includes something very similar to what I would like to implement in the stable-diffusion-webui-wd14-tagger extension that I've recently handed maintainership of. Your license permits it, so I'll pick up some of your changes, unless you disagree. Your code looks pretty good, I'll have to translate some comments I am unable to read. Thanks.

[WSH032]

Thank you for taking note of this repository Of course, please feel free to use it.😊

[picobyte]

Thank you!

[WSH032]

Is that you want to display the sd-webui-infinite-image-browsing in gradio.Tab? This is the relevant section: https://github.com/WSH032/sd-webui-fast-dataset-maker/blob/5bc34f3edd1d6f4040f77cd94db3919b60b10a92/extensions/extensions_ui.py#L49

---

In fact, sd-webui-fast-dataset-maker simulates the startup behavior of A1111 WebUI

If you just want to display sd-webui-infinite-image-browsing in gradio.Tab, you should refer to https://github.com/zanllp/sd-webui-infinite-image-browsing/blob/b29242efc0dc72044e4b7ed439888841ed3ad178/scripts/iib_setup.py#L16

[picobyte]

Thanks. It seems that you have the extensions as git submodules, maybe that is the only option, but I was initally thinking about using the extensions from the parent directory. like so, if that is possible, I don't know yet.

[WSH032]

Not very understanding Do you mean importing another independent extension sd-webui-infinite-image-browsing in stable-diffusion-webui-wd14-tagger ?

I don't think so, because users may change the folder name of iib, so you won't be able to find and import it. If you really want to embed iib into stable-diffusion-webui-wd14-tagger, I think it's best to control folder names through submodules

---

I noticed that you seem to be implementing the images deduplication function?

And, you want to display duplicate images in gradio?

[picobyte]

You are probably right and a submodule avoids conflicts. I am thinking about several options:

1. Duplicates, correct, in particular also removing or moving files if they are, but a bit more than simple duplicate search; also a highly similar search within the set of images for a batch query that has been tagged.
2. Sorting images based on selected tags and weights for the images that have been batch queried.
3. Some file management (moving files) so that more images can be added to the batch query folder for tagging. From a gallery like yours on a tab, images can be interrogated (tagged) easily by a drag-drop.

It's a lot but I think it could be beneficial.

[WSH032]

Cool !

---

Duplicates, correct, in particular also removing or moving files if they are, but a bit more than simple duplicate search; also a highly similar search within the set of images for a batch query that has been tagged.

I have an extension for deduplication.

My idea is to mark them by renaming, or move these duplicate images to another folder, and then view and process them through iib.

But I don't embed iib into this deduplication extension, but rather adapt to independently running iib by modifying the deduplication extension

---

supplement:

I provide users with easy viewing and processing methods through Gradio.Gallery, rather than through iib

If users need more advanced processing, I will provide move or rename functions to mark these images for use in iib

[WSH032]

Thanks. It seems that you have the extensions as git submodules, maybe that is the only option, but I was initally thinking about using the extensions from the parent directory. like so, if that is possible, I don't know yet.

If you really want to embed iib into stable-diffusion-webui-wd14-tagger, it seems that you don't need to import iib

iib seems to rely on FastAPI and JS to work, which means that if the user has already installed iib extension, you don't need to import it again when use in A1111-WebUI. You just need to create your own Gradio components and keep the same elem_id.

gr.HTML("error", elem_id="infinite_image_browsing_container_wrapper")

I am not sure, I will try later

---

supplement:

Okay, it does work, but don't do that, it will invalidate the original iib

Perhaps we should ask the author of iib

[picobyte]

Thanks for the information. I've already started a bit, but it will take some time, before I understand and then I may have questions. Don't expect this to be included any time soon. It's a slow process. I'll first make a feature branch for this. That's for sure.

[WSH032]

Looking forward to it😀

If there is anything I can help, I would be happy to

stable-diffusion-webui-wd14-tagger is a very useful extension that greatly helped me in SD-lora-training. Thank you for your work

[picobyte]

I have just started, and I've already made tons of changes. Though I do a lot of testing, I hope I didn't also break things for others. There were some issues already, e.g. fastapi, but I think I resolved them also. we'll see.. Thanks for the heads up.

[picobyte]

The feature branch I started with is here Its addition is just the extensions directory, with English added and some white space changes. I'm not even sure if I'm going to need all plugins, that's for later. If you like to help, you are welcome of course.

[WSH032]

Awsome, these English translations are correct

But there is one thing to note. sd-webui-fast-dataset-maker only supports running independently of A1111-WebUI, do you know this?

Is that you want to embed iib here when use in A1111-WebUI?

If so, there would be a better implementation method instead of relying on sd-webui-fast-dataset-make

[picobyte]

ok, thanks a lot! yes, I'd like to run it on the tab, like you point out. you wrote about this before, I'll check it out. with lib you mean the infinite image browser? Or you mean that I can just import this as a library

[WSH032]

with lib you mean the infinite image browser?

yes, iib == infinite image browser

---

I'd like to run it on the tab, like you point out.

In fact, sd-webui-fast-dataset-maker only provides a runtime environment similar to A1111-WebUI for iib, rather than embedding iib into specific tabs

I am sure there is a better way to run iib on the tab of own extension. I will communicate with the author of iib to see if it is possible.

[picobyte]

Well maybe a standalone version is a very interesting idea as well, a second app altogether, but currently I think I'll keep this an automatic1111 extension to keep it a little simple for myself. It's already a lot of changes. If you'd like to include some of the tagging features, then that would be an option as well.

[WSH032]

I'll keep this an automatic1111 extension to keep it a little simple for myself.

I agree with you. As an extension, it will be more flexible

---

like this?

[picobyte]

Yes, exactly like that.

[WSH032]

That's it, dont change elem_id

---

But it will make the original iib unusable

---

supplement:

But why not let users install an iib extension themselves

[picobyte]

ok I'm trying it out now.

It just says error there.

[WSH032]

Oh, sorry. forgive me, I overlooked that implementing this requires installing iib

[picobyte]

ah ok you install it as a submodule, or an import?

[WSH032]

ah ok you install it as a submodule?

Of course, this can be achieved, but it still conflicts with the original iib I will ask the author of iib

[picobyte]

I could just fork the imagebrowser, clone my fork as a submodule and change the elem_ids in javascript, css and ui

[WSH032]

This is a feasible solution

Do you konw about how fastapi and script_callbacks.on_app_started work？

As far as I know, iib needs to use this callback to register fastapi

If the user installs the original iib at the same time, I am not sure if there is a conflict between the two callbacks

---

And, this is javascript of iib

• This is a file generated using yarn build.
• If you want to make changes, please modify index.tpl.js and run the command to generate it again.

[picobyte]

I recently learned about the tagger included a fastapi, I didn;t know before and that was what I broke with my changes, but I believe I've fixed it. I also added a remote unload function. I don't know yet about script_callbacks.on_app_started, but I'm sure that I can also find out, thanks for all the information, it is very kind of you.

I will test it out, of course, I have the extension as well.

[WSH032]

That's how A1111-WebUI script_callbacks.on_app_started work https://github.com/WSH032/sd-webui-fast-dataset-maker/blob/5bc34f3edd1d6f4040f77cd94db3919b60b10a92/webui.py#L138

May be useful for your understanding

---

I still think it would be better for users to install iib themselves

The duplicate check function only needs to provide users with a simple gallery, or mark images for independent iib to use

[picobyte]

Also I believe your repository would be useful: image-deduplicate-cluster-webui I also implemented something very simple myself, but it's just based on sorted tags + weights with a precision of one digit. That already was kind of working, though and allowing me to find duplicates slight crop, horizontal flip, file rename and rescaled image. It does also list some highly similar photos. But your implementation is more advanced, and why reinvent the wheel?

[WSH032]

Thank you for your comment!

But your implementation is more advanced

I didn't write many algorithms. I just used the imagededup library to check the duplicate and display the results in the gallery

---

But it is better not to refer to it first. The code of version 1.0.0 is too bad. I am reconstructing it. I will release version 2.0.0 when I am free, probably the day after tomorrow

---

supplement:

that's alpha for reconstructing https://github.com/WSH032/image-deduplicate-cluster-webui/tree/dev-2.0.0-alpha.1

[WSH032]

Hi, check this commit

I almost finished it. But this still conflicts with the original iib, and it may not work if the user has already installed iib. Don’t worry about this, in the last discussion, the author of iib said he would solve the elem_id problem.