[Bug]: Everytime I "Interrogate CLIP" it gives error

[SingingTurnip]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

I moved an image from txt2img to inpaint and pressed on Interrogate CLIP

Steps to reproduce the problem

1. Use model v1-5-pruned-emaonly.ckpt
2. Go to txt2img make size 512x768
3. Use the prompt a beautiful portrait of a cute cyberpunk dog by greg rutkowski and wlop and sandra chevrier, purple blue color scheme, high key lighting, volumetric light, digital art, highly detailed, fine detail, intricate, ornate, complex, octane render, unreal engine, photorealistic
4. Use Huen
5. Use the seed 1156649257
6. Wait until it is done and choose the image with 2 dog heads and move it to inpaint
7. Press Interrogate Clip

What should have happened?

It should've comeback with what it could see rather than the previous prompt and red error text.

Commit where the problem happens

99d728b5b18829c8a6b7b2d69c9b9327dd257896

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Microsoft Edge

Command Line Arguments

No response

Additional information, context and logs

``Error interrogating Traceback (most recent call last): File "C:\SD\stable-diffusion-webui\modules\interrogate.py", line 143, in interrogate caption = self.generate_caption(pil_image) File "C:\SD\stable-diffusion-webui\modules\interrogate.py", line 121, in generate_caption gpu_image = transforms.Compose([ File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\transforms.py", line 94, in call img = t(img) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\transforms.py", line 349, in forward return F.resize(img, self.size, self.interpolation, self.max_size, self.antialias) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\functional.py", line 430, in resize return F_pil.resize(img, size=size, interpolation=pil_interpolation, max_size=max_size) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\functional_pil.py", line 249, in resize raise TypeError(f"img should be PIL Image. Got {type(img)}") TypeError: img should be PIL Image. Got <class 'NoneType'>

Traceback (most recent call last): File "C:\SD\stable-diffusion-webui\modules\interrogate.py", line 143, in interrogate caption = self.generate_caption(pil_image) File "C:\SD\stable-diffusion-webui\modules\interrogate.py", line 121, in generate_caption gpu_image = transforms.Compose([ File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\transforms.py", line 94, in call img = t(img) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\transforms.py", line 349, in forward return F.resize(img, self.size, self.interpolation, self.max_size, self.antialias) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\functional.py", line 430, in resize return F_pil.resize(img, size=size, interpolation=pil_interpolation, max_size=max_size) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\functional_pil.py", line 249, in resize raise TypeError(f"img should be PIL Image. Got {type(img)}") TypeError: img should be PIL Image. Got <class 'NoneType'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 275, in run_predict output = await app.blocks.process_api( File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 787, in process_api result = await self.call_function(fn_index, inputs, iterator) File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 694, in call_function prediction = await anyio.to_thread.run_sync( File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "c:\SD\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, *args) File "C:\SD\stable-diffusion-webui\modules\ui.py", line 387, in interrogate prompt = shared.interrogator.interrogate(image) File "C:\SD\stable-diffusion-webui\modules\interrogate.py", line 173, in interrogate res += "" TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

[Luke-Beck]

I've gotten this before as well. But I just realised that Interrogate uses the image from the img2img/img2img tab and not the image in the img2img/inpaint tab. I'm assuming there is no image loaded into your img2img/img2img tab.

[BaptisteGarcin]

same and I have loaded an img

[VspiderXV]

Getting the same error, there is an img loaded in the img2img/img2img tab, tried the solution from thread #2848 and still getting the same error.

[jtoy]

fixed?

[Lalimec]

I have the same issue.

[VspiderXV]

Hi, at least in my case the fix was to re-install transformers==4.19.2 per the "requirements-txt" file, at some point I had upgraded to a newer version and that was causing the issue, uninstall your version, install the one stated on that file and try launching webui again

[githubUser01946]

I have the same issue.

[githubUser01946]

Hi, at least in my case the fix was to re-install transformers==4.19.2 per the "requirements-txt" file, at some point I had upgraded to a newer version and that was causing the issue, uninstall your version, install the one stated on that file and try launching webui again

What are the steps you took for this?

[drsmog]

try to check folder - stable-diffusion-webui/models/BLIP if its empty install that model and after that it should work to install just download it with wget or drag and drop file in that folder

so here is instruction on Linux/Unix

$ cd stable-diffusion-webui/models/BLIP
$ wget -c https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_caption_capfilt_large.pth

[Mirdgard]

So, I've had this problem. First, I tried this solution https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2848 from BUSH48228 commented on Oct 27. It was first step, because it was part of my problem -- access to this url was denied for me. I saw that it worked because of these lines in "webui-user": Downloading: "Git/stable-diffusion-webui/repositories/BLIP/models/model_base_caption_capfilt_large.pth" to \Git\stable-diffusion-webui\models\BLIP\model_base_caption_capfilt_large.pth

But the problem remained. So like drsmog commented before me, I've put file "model_base_caption_capfilt_large.pth" to the directory "Git\stable-diffusion-webui\models\BLIP\models" and it's solved my problem!

[kkiqq22]

same as me, is not fixed

[Mirdgard]

same as me, is not fixed

Did you read my comment? I wrote that you need to download the file and put it in the correct directory. I fixed this problem on my computer.

[imranmu]

There is a solution link It's the transformers library. Please check out the dev branch or modify the transformers line of extensions/sd_dreambooth_extension/requirements.txt to be

transformers==4.26.1