sd-webui-controlnet icon indicating copy to clipboard operation
sd-webui-controlnet copied to clipboard

[Bug]: Clip_vision preprocessor unnecessary memory usage/leak

Open tkalayci71 opened this issue 1 year ago • 0 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

/clip/init.py file should use torch.no_grad()

with torch.no_grad():
    clip_vision_model = clip_vision_model.to(devices.get_device_for("controlnet"))
    style_for_clip = clip_proc(images=img, return_tensors="pt")['pixel_values']
    style_feat = clip_vision_model(style_for_clip.to(devices.get_device_for("controlnet")))['last_hidden_state']

without it I had to use --lowvram, now works fine.

Steps to reproduce the problem

  1. Go to ....
  2. Press ....
  3. ...

What should have happened?

...

Commit where the problem happens

webui: controlnet: .

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

..

Console logs

...

Additional information

No response

tkalayci71 avatar Apr 07 '23 17:04 tkalayci71