Artifacts in images generated by quantized int8 (originally Fp32)

[RSMNYS]

Hi everyone!

Here are images generated by different models. The label “FP32&UNet_FP16” means that all models used FP32 precision except for the UNet, which was in FP16.

You can check them out here: https://drive.google.com/drive/folders/1_3iGbf7HSzQEdNORd5h7j_GaA_v8WAwe?usp=share_link

Looking forward to your insights!

[freedomtan]

Note: what annotated as fp32 should be dynamic quantized one (quantized int8).

Simply replace the text encoder with fp16 one, that is (fp16, dynamic quant, dynamic quant) for (text encoder, unet, vae decoder), then we can get acceptable visual quality.

Please just replace the text encoder, but we can put all the 3 fp16 tflite files to the bucket.

@anhappdev: please help upload those modes.

[RSMNYS]

Hi guys! @anhappdev here is the folder with the fp16 models: https://drive.google.com/drive/folders/11Z6OL7cUYAdYHkXWuf5kE0KU0PV6m1xa?usp=sharing

BTW, since in the app for SD pipeline we use the model path to get our models, like: std::string text_encoder_path = std::string(model_path) + "/sd_text_encoder_dynamic.tflite" - we have 2 options here:

1. Append "_fp16" to the models, and "_int8" to the original one.
2. Or have 2 folders within https://github.com/mlcommons/mobile_models/releases/download/v4.1-tflite: int8, fp16, and then use the folder we need in the app itself, like: std::string(model_path) + "/int8/sd_text_encoder_dynamic.tflite, std::string(model_path) + "/fp16/sd_text_encoder_dynamic.tflite.

Let me know guys what do you think.

[freedomtan]

Hi guys! @anhappdev here is the folder with the fp16 models: https://drive.google.com/drive/folders/11Z6OL7cUYAdYHkXWuf5kE0KU0PV6m1xa?usp=sharing

BTW, since in the app for SD pipeline we use the model path to get our models, like: std::string text_encoder_path = std::string(model_path) + "/sd_text_encoder_dynamic.tflite" - we have 2 options here:

1. Append "_fp16" to the models, and "_int8" to the original one.
2. Or have 2 folders within https://github.com/mlcommons/mobile_models/releases/download/v4.1-tflite: int8, fp16, and then use the folder we need in the app itself, like: std::string(model_path) + "/int8/sd_text_encoder_dynamic.tflite, std::string(model_path) + "/fp16/sd_text_encoder_dynamic.tflite.

Let me know guys what do you think.

I don't think we should hard-code filename in our C/C++ code. Preferably, we should make them configurable in setting files.

[anhappdev]

@RSMNYS, please create a PR for the changes you'd like to implement. Once that's done, I'll replace the URL with the one from MLCommons's cloud bucket.

[freedomtan]

@RSMNYS please send a PR to continue the merge.

[anhappdev]

@RSMNYS You can use custom_setting for model file name, please see https://github.com/mlcommons/mobile_app_open/pull/924

[RSMNYS]

Hi guys! I've added custom_settings to have models name, and adjusted stable_diffusion_pipeline. And now a question, since we are going to use int8 and fp16 models in parallel, which way to store the models would you prefer? Have the dedicated folders like fp16, int8? or have the suffix for each model like sd_decoder_dynamic_fp16?

Thanks!

BTW, I've started to use fp16 image_decoder with other int8 models, since it gives better results.

@anhappdev @freedomtan

[anhappdev]

I would prefer adding a suffix.

[freedomtan]

closed by https://github.com/mlcommons/mobile_app_open/commit/d4338bb444cf59a5616b045631019c0f1c147ffc