[Issue] Errors with RealESRGAN 4x+ Anime6B on OpenVINO backend

[kaetemi]

Issue Description

Upscale an image twice in a row, and the output will have repeated or shifted tiles. Many errors in the console. API returns broken image as if successful. The first generation is sometimes OK, though.

Version Platform Description

Ubuntu

Relevant log output

05:22:24-670650 INFO     Upscaler loaded: type=RealESRGAN 4x+ Anime6B model=models/RealESRGAN/RealESRGAN_x4plus_anime_6B.pth
05:22:24-677140 INFO     Upscaler compile: backend=openvino_fx available=['cudagraphs', 'inductor', 'onnxrt', 'openvino_fx', 'openxla', 'openxla_eval', 'tvm']
05:22:24-682806 INFO     Upscaler compile: time=0.00
Upscaling ━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━  40% 0:00:30 0:00:1905:22:44-340818 ERROR    Upscale error: type=R-ESRGAN Exception from src/inference/src/infer_request.cpp:111:                                                 
                         Exception from src/inference/src/infer_request.cpp:74:
                         Exception from src/plugins/intel_cpu/src/infer_request.cpp:393:
                         Can't set the input tensor with name: input0, because the model input (shape=[1,64,208,200]) and the tensor (shape=(1.64.208.208)) are incompatible



05:22:44-345614 ERROR    Upscale error: type=R-ESRGAN Exception from src/inference/src/infer_request.cpp:111:
                         Exception from src/inference/src/infer_request.cpp:74:
                         Exception from src/plugins/intel_cpu/src/infer_request.cpp:393:
                         Can't set the input tensor with name: input0, because the model input (shape=[1,64,208,200]) and the tensor (shape=(1.64.208.146)) are incompatible



Upscaling ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━  60% 0:00:14 0:00:2005:22:45-411526 ERROR    Upscale error: type=R-ESRGAN Exception from src/inference/src/infer_request.cpp:111:                                                 
                         Exception from src/inference/src/infer_request.cpp:74:
                         Exception from src/plugins/intel_cpu/src/infer_request.cpp:393:
                         Can't set the input tensor with name: input0, because the model input (shape=[1,64,208,200]) and the tensor (shape=(1.64.208.208)) are incompatible



05:22:45-415860 ERROR    Upscale error: type=R-ESRGAN Exception from src/inference/src/infer_request.cpp:111:
                         Exception from src/inference/src/infer_request.cpp:74:
                         Exception from src/plugins/intel_cpu/src/infer_request.cpp:393:
                         Can't set the input tensor with name: input0, because the model input (shape=[1,64,208,200]) and the tensor (shape=(1.64.208.146)) are incompatible



Upscaling ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:24
/home/kaetemi/automatic/venv/lib/python3.10/site-packages/numba/np/ufunc/parallel.py:371: NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12020. The TBB threading layer is disabled.
  warnings.warn(problem)
05:22:49-556911 INFO     Saving: image="outputs/extras/*********.png" type=PNG resolution=768x1152 size=0



---

also with weird image reso:

RuntimeError: The expanded size of the tensor (768) must match the existing size (396) at non-singleton dimension 3.  Target sizes: [1, 3, 768, 768].  Tensor sizes: [3, 768, 396]

Backend

Original

Branch

Master

Model

Other

Acknowledgements

• [X] I have read the above and searched for existing issues
• [X] I confirm that this is classified correctly and its not an extension issue

[kaetemi]

Seems to work fine when disabling compilation of upscalers.

[Disty0]

Only ESRGAN is supported on OpenVINO. Disable upscaler compile and this will make it use CPU with PyTorch instead.

It should be disabled by default now: https://github.com/vladmandic/automatic/commit/66de7597d1b44e5c78ab3b1f8fa3ff1be834562a Added a note on the Wiki page under Limitations.