F64 dtype

[valkarias]

Models with F64 type tensors do not work, specifically this SD 1.5 model: https://civitai.com/models/7371/rev-animated?modelVersionId=425083

-- even after adding a --type argument, it still fails

further research only led me to this other issue: https://github.com/leejet/stable-diffusion.cpp/issues/153 the offered program by the owner sadly did not work and i didnt want to fiddle with it any further

[DanaNikifor]

And where do you see fp64?

[DanaNikifor]

It seems like everyone is trying to reduce the model and keep the quality, but here it is))

[valkarias]

It seems like everyone is trying to reduce the model and keep the quality, but here it is))

have you looked into the related issue link, it would give more context. im not reducing the model size or manipulating it in any way. it errors simply if i use it with stable-diffusion.cpp showing errors of it not recognizing certain tensors

[vmobilis]

@stduhpf, this can be fixed, maybe easily (no, it's not that easy, thank you, @stduhpf), since GGML / SD.CPP support F64 and I64 types: https://github.com/ggml-org/ggml/blob/ff9052988b76e137bcf92bb335733933ca196ac0/include/ggml.h#L379

The function https://github.com/leejet/stable-diffusion.cpp/blob/10c6501bd05a697e014f1bee3a84e5664290c489/model.cpp#L961 should be appended with:

    } else if (dtype == "F64") {
        ttype = GGML_TYPE_F64;
    } else if (dtype == "I64") {
        ttype = GGML_TYPE_I64;

I tested on SD 1.1 (https://huggingface.co/CompVis/stable-diffusion-v1-1) model, its FP16 text encoder also has i64 tensor which breaks generation.

Also, its unet tensors have "unet.*" naming, therefore line https://github.com/leejet/stable-diffusion.cpp/blob/10c6501bd05a697e014f1bee3a84e5664290c489/model.cpp#L1618 should be changed to:

        if (tensor_storage.name.find("model.diffusion_model.") == std::string::npos &&
            tensor_storage.name.find("unet.") == std::string::npos) {

Or else diffusion model weight type will be undetected (??).

But I'm not sure if these 2 fixes are enough, because SD complained about tensor: model.cpp:1907 - unknown tensor 'cond_stage_model.transformer.text_model.embeddings.position_ids | i64 | 2 [77, 1, 1, 1, 1]' in model file

It succesfully generated an image, though:

[leejet]

It has been fixed now.