Apply loss statistics from hypernetwork to textual inversion

[ccchan49]

Apply the code from #3277 to textual inversion.py. The original author is @aria1th.

(First-time user of GitHub...hope I didn't miss anything)

[aria1th]

Here is some fixes and improvements : #3486

1. statistics.stdev only works if iterable contains more than 1 items
2. line 277 and 278 had problem, it should use loss_info[key] instead of raw loss_info or improved deque

[ccchan49]

Here is some fixes and improvements : #3486

1. statistics.stdev only works if iterable contains more than 1 items
2. line 277 and 278 had problem, it should use loss_info[key] instead of raw loss_info or improved deque

Thx. I have updated the code according to yours. I am running a full training to see if everything works, and so far so good.

[aria1th]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/3538/files#diff-d3503031ef91fb35651a650f994dd8c94d405fe8e690c41817b1d095d66b1c69R208

Actually you're having duplicate lines now, lines 208 and 215 , 293 should be removed. Except that it seems okay

[ccchan49]

Thx again. I tried to interrupt the process in a few steps and got an error. I was trying to see what happened and I saw your comment. Should be good now.

[aria1th]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/3538/files#diff-d3503031ef91fb35651a650f994dd8c94d405fe8e690c41817b1d095d66b1c69R214 its still remaining mhm

[ccchan49]

My bad. I thought you mean 208 is the duplicate of 215... It turns out there are one more line down below and it's the one that should stay lol.

[AUTOMATIC1111]

There are two functions that are plain copy-pasted. You can't do that. You should strive to not just eliminate copy-pasting entirely, but also merge similar code into one function, and you instead just copy-paste existing functions.

[ccchan49]

There are two functions that are plain copy-pasted. You can't do that. You should strive to not just eliminate copy-pasting entirely, but also merge similar code into one function, and you instead just copy-paste existing functions.

Do you mean that I should call the function in hypernetwork.py instead of doing the def here? Sry that I didn't know how do it as I just started to learn python a month ago. I just google it and give it a try, but I got an error: "AttributeError: partially initialized module 'modules.hypernetworks.hypernetwork' has no attribute 'list_hypernetworks' (most likely due to a circular import)" It seems that the entire textual_inversion.py is imported into hypernetwork.py? In this case, what I should do is keep the two functions in textual_inversion.py and delete them in hypernetwork.py?

The code I used: from modules.hypernetworks.hypernetwork import statistics, report_statistics

[aria1th]

Circular dependencies is not good. Yes, In this case we have common supermodules, so you can extract those functions and put into textual_inversion.py.

[ccchan49]

There are two functions that are plain copy-pasted. You can't do that. You should strive to not just eliminate copy-pasting entirely, but also merge similar code into one function, and you instead just copy-paste existing functions.

So I extracted the two functions from hypernetworks.py to textual_inversion.py to avoid circular import. It should be good now.

[aria1th]

I think you're now missing statistics and report_statistics import in hypernetwork side, can you confirm some screenshot that both are working?

[ccchan49]

I think you're now missing statistics and report_statistics import in hypernetwork side, can you confirm some screenshot that both are working? Oh I see... Just realized that I need to change the function from report_statistics() to textual_inversion.report_statistics to make it work.

[ccchan49]

@AUTOMATIC1111 Is there something else that I need to do?