Seamless texture generation support for qwen image, z-image, and flux

[Phylliida]

Adds support for qwen image to generate seamless textures, using a --circular flag. Edit: Also includes a --circularx and --circulary if you want only tiling on those axes

Requires this PR https://github.com/ggml-org/ggml/pull/1374 (Edit: see https://github.com/ggml-org/llama.cpp/pull/16985) for ggml that adds a "circular" mode that can be used.

I also had to tweak rope so the period of the sinusods would evenly tile.

./bin/sd  --diffusion-model /home/bepis/Documents/world/models/qwen-image-lighting-8steps-V1.0-Q4_K_S.gguf  --vae /home/bepis/Documents/world/models/qwen_image_vae.safetensors  --qwen2vl /home/bepis/Documents/world/models/Qwen2.5-VL-7B-Instruct-IQ4_XS.gguf --cfg-scale 2.5 --sampling-method euler -p "onion" --circular --seed 420

[Phylliida]

Some conflicts, lemme resolve those...

[leejet]

Nice Work! Once ggml merges the circular-pad changes, I will merge this PR.

[oscarbg]

llama.cpp PR merged already!

[Phylliida]

Needs to be modified slightly bc you can do it only circular pad no circular conv

[Phylliida]

In particular like this:

__STATIC_INLINE__ struct ggml_tensor* ggml_nn_conv_2d(struct ggml_context* ctx,
                                                      struct ggml_tensor* x,
                                                      struct ggml_tensor* w,
                                                      struct ggml_tensor* b,
                                                      int s0      = 1,
                                                      int s1      = 1,
                                                      int p0      = 0,
                                                      int p1      = 0,
                                                      int d0      = 1,
                                                      int d1      = 1,
                                                      bool direct = false,
                                                      float scale = 1.f) {
    if (scale != 1.f) {
        x = ggml_scale(ctx, x, scale);
    }
    const bool use_circular = sd_is_circular_padding_enabled();
    LOG_DEBUG("use circular conv %d", use_circular ? 1 : 0);
    const bool is_depthwise = (w->ne[2] == 1 && x->ne[2] == w->ne[3]);

    if (use_circular && (p0 != 0 || p1 != 0)) {
        x  = ggml_pad_ext_circular(ctx, x, p0, p0, p1, p1, 0, 0, 0, 0);
        p0 = 0;
        p1 = 0;
    }

    if (direct) {
        if (is_depthwise) {
            x = ggml_conv_2d_dw_direct(ctx, w, x, s0, s1, p0, p1, d0, d1);
        } else {
            x = ggml_conv_2d_direct(ctx, w, x, s0, s1, p0, p1, d0, d1);
        }
    } else {
        x = ggml_conv_2d(ctx, w, x, s0, s1, p0, p1, d0, d1);
    }
    if (scale != 1.f) {
        x = ggml_scale(ctx, x, 1.f / scale);
    }
    if (b != NULL) {
        b = ggml_reshape_4d(ctx, b, 1, 1, b->ne[0], 1);
        x = ggml_add_inplace(ctx, x, b);
    }
    return x;
}

I can clean up this PR to that on wednesday

[leejet]

I think we can add a field in GGMLRunnerContext to control whether circular padding is used.

[Phylliida]

looked into this, though it was merged into llama.cpp, we'll need to wait for ggml to be synced with llama.cpp's ggml (it seems to only be synced every few weeks), or alternatively adjust the reference to point to llama.cpp's ggml folder instead of ggml

[pwilkin]

@ggerganov any chance for a GGML sync?

[ggerganov]

It's synced now - thanks

[Phylliida]

in progress, wait a few min...

[Phylliida]

Okay it is working for qwen image, should also work for other models but I haven't tested them yet, anyone else feel free to