[Feature Request]: Seamless texture SD upscale

[Aza-zeal]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

I wish to be able to use SD upscaling on seamless textures. Upscaling with the SD upscale script should maintain the original color and composition at the edges of the image when "Tiling" is ticked.

Proposed workflow

1. Upload a seamless texture to Sd upscale
2. Tick "tiling" field and generate
3. Results in an SD upscaled image that is still seamless at the edges (tileable)

Additional information

I wish to omit arduous, repetitive processes like fixing seams from the process of texture creation.

[rasamaya]

I always use upscale, after highres fix. This tile feature would be usefull. Good idea.

[jfjensen]

This is exactly what I'm looking for too.

[erdostamasa]

This would be the exact thing I need. Right now I can only use the tiling images in their original 512 * 512 resolution because upscaling introduces edge artifacts.

[erdostamasa]

Does anybody have an idea how could this be achieved?

[jfjensen]

I've found a work-around, that may even be used as a solution, if someone is willing to code it.

These are the steps:

1. generate your texture with the tileable setting turned on (result: 512x512 image)
2. tile the resulting image 2 by 2; meaning 2 tiles in X and Y direction = 4 tiles in total (result: 1024x1024 image)
3. upscale the 2-by-2 tiled image as much as you like (result: for example 4096x4096 for 4x upscaling)
4. crop the center part of the upscaled image (result in this case an 2048x2048 image)
5. check that the center crop is seamlessly tileable - which it usually is...

Like this you get a tileable version of the original image that is upscaled 4x. Do note that the resulting image is offset by width/2 and height/2. With GIMP or Photoshop you can re-offset it the image to get the original one, if you need this.

[rexelbartolome]

Ohh interesting, my workaround so far is

1. generate a tiling texture, let's call it the A-tile
2. go to Krita/GIMP/PS, offset the A-tile, now let's call it B-tile
3. upscale A-tile
4. upscale B-tile
5. go to Krita/GIMP/PS, open upscaled A-tile
6. in the same document, open upscaled B-tile and offset it
7. erase the center of B-tile (with a soft brush ideally) to your liking, since the edges are what we mainly need

but @jfjensen 's seem much simpler :)

If one wants more control, then ideally you'd be doing this process inside the painting program that's using a plugin to connect to SD. Krita, GIMP, and Photoshop I think all have their own plugins, better masking and inpainting experience overall.

[rexelbartolome]

Hello again, with the help of ChatGPT I've managed to make a script that simplifies and modifies some of the steps in @jfjensen 's workflow.

First we need ofc a tileable image

then we use this tile and crop script: this will basically extend the image by a half tile

from PIL import Image

# Path of where the source image is (use double backslash)
inpath = "image.png"
# Path of where to save the image
saveloc = "1-5x1-5.png"

# Open the image
image = Image.open(inpath)

# Get the width and height of the image
width, height = image.size

# Calculate the size of the tiled image
tiled_width = width * 3
tiled_height = height * 3

# Create a new image with the calculated size
tiled_image = Image.new(image.mode, (tiled_width, tiled_height))

# Paste the image onto the tiled image in a 3x3 grid
for i in range(3):
    for j in range(3):
        tiled_image.paste(image, (i * width, j * height))

# Calculate the center point of the tiled image
center_x = tiled_width / 2
center_y = tiled_height / 2

# Calculate the size of the cropped image
cropped_width = tiled_width / 1.5
cropped_height = tiled_height / 1.5

# Crop the tiled image using the center point as the anchor
cropped_image = tiled_image.crop((center_x - cropped_width / 2, center_y - cropped_height / 2,
                                  center_x + cropped_width / 2, center_y + cropped_height / 2))

# Save the tiled image
cropped_image.save(saveloc)

resulting image:

Then, the output image should be the one that goes into the upscaler.

After getting upscaled, you just need to use this next script to crop it back to a 1x1 tile.

from PIL import Image

# Path of where the source image is (use double backslash)
inpath = "upscaled.png"
# Path of where to save the image
saveloc = "1x1tile.png"

# Open the image
image = Image.open(inpath)

# Get the width and height of the image
width, height = image.size

# Calculate the center point of the image
center_x = width / 2
center_y = height / 2

# Calculate the size of the cropped image
cropped_width = width / 2
cropped_height = height / 2

# Crop the image using the center point as the anchor
cropped_image = image.crop((center_x - cropped_width / 2, center_y - cropped_height / 2,
                            center_x + cropped_width / 2, center_y + cropped_height / 2))

# Save the cropped image
cropped_image.save(saveloc)

Here's a closeup on the (barely noticeable) seams

https://user-images.githubusercontent.com/88552647/210046679-3b195b67-43b9-462d-bb13-9936a43a706b.mp4

Now the only thing left is I guess implementing it in a way that talks to A1111 webui which is way beyond my knowledge...

There's also no scripting section in the Extras/upscalers tab so the script should be either an extension in it's own separate tab that's basically a copy of Extras but with the extra buttons that applies the 2 scripts or maybe just a script in general but it only works in img2img under the scripts section(?)

So if any of you knows how this can be implemented, please feel free to pick up from here :D

[rexelbartolome]

If you guys want a more automated way and are okay with a bit of a learning curve to use a node system, then this will likely be the best solution https://github.com/chaiNNer-org

AFAIK it can do what my script just did, tile a texture, upscale, then crop afterwards. Can also do them in batches and you can integrate multiple upscalers to use it with instead of A1111's two upscaling layer system

[MultiUpGame]

Hello. Can't make a video? I don't understand where to write the code. Or what did you do. Thanks

[rexelbartolome]

For @MultiUpGame and anyone still wondering how to do it I have made a video about it https://youtu.be/sUyWWjFlszY