progrockdiffusion icon indicating copy to clipboard operation
progrockdiffusion copied to clipboard

Improving gobig

Open Letiliel opened this issue 3 years ago • 8 comments

Gobig is absolutely awesome but the overlapping parts are mushy and I hope it can be improved. The blur is at least half due to the fact DD renders less details on the sides of the image (especially so with higher cut_ic_pow I guess), so the situation could be improved by increasing the size of the overlapping parts and discarding a portion of them before blending?

Also, it'd be nice to have a setting for the number of slices.

Letiliel avatar Jul 02 '22 12:07 Letiliel

Great thoughts. Let me ponder this!

I'm not sure DD always renders less details on the sides. I've seen plenty of images with excellent detail across the whole thing, so I suspect this might be a perceived problem that is more due to settings/prompts?

As for slices, can you help me understand why you would want to be able to adjust this? The number is calculated based on the relation between the original image size and the final image size, to ensure that no slice is bigger (in total pixels) than the original so that out-of-memory errors don't occur. So, I just want to understand the need, as it could be that some other value that can be tweaked would make more sense.

lowfuel avatar Jul 05 '22 23:07 lowfuel

I suspect this might be a perceived problem that is more due to settings/prompts?

It's possible. But it's actually difficult to notice it on standard renders (I never did before realizing it with the gobig renders), it feels normal to have less details on the sides of the image and there's often a bit of depth of field that hides the "issue". That there is less details is also coherent with that heatmap visualization of cuts, that seems to show there are more cuts done towards the center (if it's the same with DD): https://media.discordapp.net/attachments/946925516219371630/972904961438543912/unknown.png

For slices, I need to test again to be sure but I've had OOM errors and fixed that by increasing the number of slices in the code. And also to fix the less-detailed-sides I've rendered the same image with a different number of slices and then blended the good parts in Gimp.

Letiliel avatar Jul 06 '22 16:07 Letiliel

Interesting! Okay, thanks for the info. I wonder if contributing to that as well is that most of the images the models are trained on have their subject matter / detail in the center. I'll look into this more.

Also I will add a setting for slices.

lowfuel avatar Jul 07 '22 01:07 lowfuel

The command line option for slices is in:

--gobig_slices 6

But please note that there is still a calculation that occurs which may add an additional slice. This is because of the need for everything to be divisible by 64, so when dividing your original image into slices, and factoring in the overlap needed, sometimes the rounding requires an extra slice to be added just to make sure it's all covered.

lowfuel avatar Jul 09 '22 17:07 lowfuel

Also, btw, another contributor is looking into improving the heatmap pattern of innercuts. Nothing concrete yet but I think it's a promising start. We'll keep you posted.

lowfuel avatar Jul 10 '22 00:07 lowfuel

Excellent, thank you!

Letiliel avatar Jul 10 '22 11:07 Letiliel

Hey again! The new method for calculating innercut locations is in now, and working well! Give it a try. It's on by default so there's nothing to change.

Also wanted to mention: In my experiments one thing I've learned is there's no magic set of settings for gobig. Each image (or at least style of image) will likely need some tweaks to get things as good as they can be. The main things to tweak are gobig_skip_ratio (I find between .5 and .65 works, but where you are in that range has a big impact), and tv_scale.

Also helpful is to pre-scale your image in photoshop (or paint.net, gimp, photopea, etc) and use the --gobiginit_scaled option to tell it you've already scaled your image. The built-in scaling method is okay, but can leave some lines that are aliased, which the Disco renderer likes to leave alone for some reason. A smoother scale (even very slightly blurry) is actually better.

lowfuel avatar Jul 15 '22 13:07 lowfuel

Awesome! And thanks for the tips :)

Letiliel avatar Jul 16 '22 14:07 Letiliel