sd-webui-deforum
sd-webui-deforum copied to clipboard
deforum-art
[Bug]: extract_to_frame can not be higher than extract_from_frame
Have you read the latest version of the FAQ?
- [X] I have visited the FAQ page right now and my issue is not present there
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui
Are you using the latest version of the Deforum extension?
- [X] I have Deforum updated to the lastest version and I still have the issue.
What happened?
Exception thrown when initially rendering with only contorl nets:
webui/scripts/deforum_helpers/video_audio_utilities.py", line 65, in vid2frames raise RuntimeError('Error: extract_to_frame can not be higher than extract_from_frame') RuntimeError: Error: extract_to_frame can not be higher than extract_from_frame
Steps to reproduce the problem
Set Hybrid Video -> Composite and Motion to None Have at least one ControlNet, with the correct video path Set Keyframes -> Max Frames to -1
As far as I can tell, all other settings can be changed (including init -> extract_to and extract_from, which can both be a number or for extract to -1). It should yield something akin to the attached in the CLI. cli_out.txt
What should have happened/how would you fix it?
All the frames for the init video should have been extracted to start off the Diffusion process.
I think this error is a result of the Keyframes -> Max Frames set to -1, as if this is a positive int it works fine. Also switching Hybrid Video -> Composite -> Normal and having Keyframes -> Max Frames -1 results in no issues. (Composite has to be None and max frames -1 for the issue to happen) Hope this helps!
Torch version
python: 3.10.12 torch: 2.0.0+cu118
On which platform are you launching the webui with the extension?
Google Colab (Other)
Deforum settings
Webui core settings
ui-config.zip This box does not accept .json files, so Its attached within a .zip.
Console logs
/content
Mounted at /content/drive
env: PYTHONDONTWRITEBYTECODE=1
env: TF_CPP_MIN_LOG_LEVEL=1
28 packages can be upgraded. Run 'apt list --upgradable' to see them.
--2023-09-14 13:13:33-- https://github.com/camenduru/gperftools/releases/download/v1.0/libtcmalloc_minimal.so.4
Resolving github.com (github.com)... 20.205.243.166
Connecting to github.com (github.com)|20.205.243.166|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/669786276/620e2e64-be9f-4599-904f-18ee3811e159?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230914%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230914T131333Z&X-Amz-Expires=300&X-Amz-Signature=6508c5da845f11d3ce1b406b2814553448122d7384bf23b03e2d6617e032531a&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=669786276&response-content-disposition=attachment%3B%20filename%3Dlibtcmalloc_minimal.so.4&response-content-type=application%2Foctet-stream [following]
--2023-09-14 13:13:33-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/669786276/620e2e64-be9f-4599-904f-18ee3811e159?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230914%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230914T131333Z&X-Amz-Expires=300&X-Amz-Signature=6508c5da845f11d3ce1b406b2814553448122d7384bf23b03e2d6617e032531a&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=669786276&response-content-disposition=attachment%3B%20filename%3Dlibtcmalloc_minimal.so.4&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 373960 (365K) [application/octet-stream]
Saving to: ‘/content/libtcmalloc_minimal.so.4’
/content/libtcmallo 100%[===================>] 365.20K --.-KB/s in 0.005s
2023-09-14 13:13:33 (67.8 MB/s) - ‘/content/libtcmalloc_minimal.so.4’ saved [373960/373960]
env: LD_PRELOAD=/content/libtcmalloc_minimal.so.4
pkg-config is already the newest version (0.29.2-1ubuntu3).
python3-dev is already the newest version (3.10.6-1~22.04).
python3-dev set to manually installed.
The following additional packages will be installed:
libaria2-0 libblkid-dev libc-ares2 libcairo-script-interpreter2 libffi-dev
libglib2.0-dev libglib2.0-dev-bin libice-dev liblzo2-2 libmount-dev
libpixman-1-dev libselinux1-dev libsepol-dev libsm-dev libxcb-render0-dev
libxcb-shm0-dev
Suggested packages:
libcairo2-doc libgirepository1.0-dev libglib2.0-doc libgdk-pixbuf2.0-bin
| libgdk-pixbuf2.0-dev libxml2-utils libice-doc libsm-doc
The following NEW packages will be installed:
aria2 libaria2-0 libblkid-dev libc-ares2 libcairo-script-interpreter2
libcairo2-dev libffi-dev libglib2.0-dev libglib2.0-dev-bin libice-dev
liblzo2-2 libmount-dev libpixman-1-dev libselinux1-dev libsepol-dev
libsm-dev libxcb-render0-dev libxcb-shm0-dev
0 upgraded, 18 newly installed, 0 to remove and 28 not upgraded.
Need to get 5,352 kB of archives.
After this operation, 25.2 MB of additional disk space will be used.
Selecting previously unselected package libc-ares2:amd64.
(Reading database ... 120901 files and directories currently installed.)
Preparing to unpack .../00-libc-ares2_1.18.1-1ubuntu0.22.04.2_amd64.deb ...
Unpacking libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.2) ...
Selecting previously unselected package libaria2-0:amd64.
Preparing to unpack .../01-libaria2-0_1.36.0-1_amd64.deb ...
Unpacking libaria2-0:amd64 (1.36.0-1) ...
Selecting previously unselected package aria2.
Preparing to unpack .../02-aria2_1.36.0-1_amd64.deb ...
Unpacking aria2 (1.36.0-1) ...
Selecting previously unselected package liblzo2-2:amd64.
Preparing to unpack .../03-liblzo2-2_2.10-2build3_amd64.deb ...
Unpacking liblzo2-2:amd64 (2.10-2build3) ...
Selecting previously unselected package libcairo-script-interpreter2:amd64.
Preparing to unpack .../04-libcairo-script-interpreter2_1.16.0-5ubuntu2_amd64.deb ...
Unpacking libcairo-script-interpreter2:amd64 (1.16.0-5ubuntu2) ...
Selecting previously unselected package libice-dev:amd64.
Preparing to unpack .../05-libice-dev_2%3a1.0.10-1build2_amd64.deb ...
Unpacking libice-dev:amd64 (2:1.0.10-1build2) ...
Selecting previously unselected package libsm-dev:amd64.
Preparing to unpack .../06-libsm-dev_2%3a1.2.3-1build2_amd64.deb ...
Unpacking libsm-dev:amd64 (2:1.2.3-1build2) ...
Selecting previously unselected package libpixman-1-dev:amd64.
Preparing to unpack .../07-libpixman-1-dev_0.40.0-1ubuntu0.22.04.1_amd64.deb ...
Unpacking libpixman-1-dev:amd64 (0.40.0-1ubuntu0.22.04.1) ...
Selecting previously unselected package libxcb-render0-dev:amd64.
Preparing to unpack .../08-libxcb-render0-dev_1.14-3ubuntu3_amd64.deb ...
Unpacking libxcb-render0-dev:amd64 (1.14-3ubuntu3) ...
Selecting previously unselected package libxcb-shm0-dev:amd64.
Preparing to unpack .../09-libxcb-shm0-dev_1.14-3ubuntu3_amd64.deb ...
Unpacking libxcb-shm0-dev:amd64 (1.14-3ubuntu3) ...
Selecting previously unselected package libffi-dev:amd64.
Preparing to unpack .../10-libffi-dev_3.4.2-4_amd64.deb ...
Unpacking libffi-dev:amd64 (3.4.2-4) ...
Selecting previously unselected package libglib2.0-dev-bin.
Preparing to unpack .../11-libglib2.0-dev-bin_2.72.4-0ubuntu2.2_amd64.deb ...
Unpacking libglib2.0-dev-bin (2.72.4-0ubuntu2.2) ...
Selecting previously unselected package libblkid-dev:amd64.
Preparing to unpack .../12-libblkid-dev_2.37.2-4ubuntu3_amd64.deb ...
Unpacking libblkid-dev:amd64 (2.37.2-4ubuntu3) ...
Selecting previously unselected package libsepol-dev:amd64.
Preparing to unpack .../13-libsepol-dev_3.3-1build1_amd64.deb ...
Unpacking libsepol-dev:amd64 (3.3-1build1) ...
Selecting previously unselected package libselinux1-dev:amd64.
Preparing to unpack .../14-libselinux1-dev_3.3-1build2_amd64.deb ...
Unpacking libselinux1-dev:amd64 (3.3-1build2) ...
Selecting previously unselected package libmount-dev:amd64.
Preparing to unpack .../15-libmount-dev_2.37.2-4ubuntu3_amd64.deb ...
Unpacking libmount-dev:amd64 (2.37.2-4ubuntu3) ...
Selecting previously unselected package libglib2.0-dev:amd64.
Preparing to unpack .../16-libglib2.0-dev_2.72.4-0ubuntu2.2_amd64.deb ...
Unpacking libglib2.0-dev:amd64 (2.72.4-0ubuntu2.2) ...
Selecting previously unselected package libcairo2-dev:amd64.
Preparing to unpack .../17-libcairo2-dev_1.16.0-5ubuntu2_amd64.deb ...
Unpacking libcairo2-dev:amd64 (1.16.0-5ubuntu2) ...
Setting up libglib2.0-dev-bin (2.72.4-0ubuntu2.2) ...
Setting up libblkid-dev:amd64 (2.37.2-4ubuntu3) ...
Setting up libpixman-1-dev:amd64 (0.40.0-1ubuntu0.22.04.1) ...
Setting up libice-dev:amd64 (2:1.0.10-1build2) ...
Setting up libsm-dev:amd64 (2:1.2.3-1build2) ...
Setting up liblzo2-2:amd64 (2.10-2build3) ...
Setting up libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.2) ...
Setting up libffi-dev:amd64 (3.4.2-4) ...
Setting up libxcb-shm0-dev:amd64 (1.14-3ubuntu3) ...
Setting up libsepol-dev:amd64 (3.3-1build1) ...
Setting up libxcb-render0-dev:amd64 (1.14-3ubuntu3) ...
Setting up libcairo-script-interpreter2:amd64 (1.16.0-5ubuntu2) ...
Setting up libaria2-0:amd64 (1.36.0-1) ...
Setting up libselinux1-dev:amd64 (3.3-1build2) ...
Setting up aria2 (1.36.0-1) ...
Setting up libmount-dev:amd64 (2.37.2-4ubuntu3) ...
Setting up libglib2.0-dev:amd64 (2.72.4-0ubuntu2.2) ...
Processing triggers for libglib2.0-0:amd64 (2.72.4-0ubuntu2.2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link
/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link
Processing triggers for man-db (2.10.2-1) ...
Setting up libcairo2-dev:amd64 (1.16.0-5ubuntu2) ...
The following NEW packages will be installed:
unionfs-fuse
0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
Need to get 48.7 kB of archives.
After this operation, 146 kB of additional disk space will be used.
Selecting previously unselected package unionfs-fuse.
(Reading database ... 121748 files and directories currently installed.)
Preparing to unpack .../unionfs-fuse_1.0-1ubuntu2_amd64.deb ...
Unpacking unionfs-fuse (1.0-1ubuntu2) ...
Setting up unionfs-fuse (1.0-1ubuntu2) ...
Processing triggers for man-db (2.10.2-1) ...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 2.3/2.3 GB 185.4 MB/s eta 0:00:01tcmalloc: large alloc 2267275264 bytes == 0x5c78810a8000 @
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 2.3/2.3 GB 168.9 MB/s eta 0:00:01tcmalloc: large alloc 2834096128 bytes == 0x5c79082e6000 @
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 2.3/2.3 GB 168.9 MB/s eta 0:00:01tcmalloc: large alloc 2267275264 bytes == 0x5c78810a8000 @
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 GB 890.0 kB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.1/6.1 MB 82.8 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 59.3 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 47.1 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 81.2 MB/s eta 0:00:00
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 108.2/108.2 MB 14.1 MB/s eta 0:00:00
/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui
Updating files: 100% (203/203), done.
HEAD is now at 22bcc7be attempted fix for infinite loading for settings that some people experience
Updating files: 100% (135/135), done.
HEAD is now at cf1d67a Update modelcard.md
HEAD is now at c202932 GRADIO_TUNNEL online
Already up to date.
Download Results:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
d3460e|OK | 0B/s|/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/models/CLIP/ViT-L-14.pt
Status Legend:
(OK):download completed.
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
Installing gfpgan
Installing clip
Installing open_clip
Installing requirements for CodeFormer
Installing requirements for Web UI
Installing Deforum requirement: av
Installing Deforum requirement: pims
Installing sd-webui-controlnet requirement: mediapipe
Installing sd-webui-controlnet requirement: svglib
Installing sd-webui-controlnet requirement: fvcore
Installing pycloudflared
Launching Web UI with arguments: --listen --xformers --enable-insecure-extension-access --theme dark --gradio-queue --clip-models-path /content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/models/CLIP --ckpt-dir /content/fused-models --lora-dir /content/fused-lora --multiple
2023-09-14 13:20:00.730518: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/usr/local/lib/python3.10/dist-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
warnings.warn(
Additional Network extension not installed, Only hijack built-in lora
LoCon Extension hijack built-in lora successfully
2023-09-14 13:22:10,324 - ControlNet - INFO - ControlNet v1.1.234
ControlNet preprocessor location: /content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
2023-09-14 13:22:27,411 - ControlNet - INFO - ControlNet v1.1.234
all detected, remote.moe trying to connect...
Warning: Permanently added 'localhost.run' (RSA) to the list of known hosts.
all detected, cloudflared trying to connect...
Download cloudflared...: 100% 34.8M/34.8M [00:00<00:00, 507MB/s]
Loading weights [4199bcdd14] from /content/fused-models/revAnimated_v122.safetensors
Creating model from config: /content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Downloading (…)olve/main/vocab.json: 100% 961k/961k [00:00<00:00, 4.13MB/s]
Downloading (…)olve/main/merges.txt: 100% 525k/525k [00:00<00:00, 52.3MB/s]
Downloading (…)cial_tokens_map.json: 100% 389/389 [00:00<00:00, 2.32MB/s]
Downloading (…)okenizer_config.json: 100% 905/905 [00:00<00:00, 5.83MB/s]
Downloading (…)lve/main/config.json: 100% 4.52k/4.52k [00:00<00:00, 22.7MB/s]
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(10): badhandv4, verybadimagenegative_v1.3, bad_prompt_version2, EasyNegative, bad-image-v2-39000, EasyNegativeV2, bad-hands-5, bad-artist, bad-artist-anime, ng_deepnegative_v1_75t
Model loaded in 154.0s (load weights from disk: 85.8s, create model: 5.1s, apply weights to model: 34.4s, load textual inversion embeddings: 28.6s).
*Deforum ControlNet support: enabled*
Public WebUI Colab URL: None
Public WebUI Colab URL: https://2ba806d6-11bf-443f.gradio.live
Public WebUI Colab URL: https://1addd794d95072.lhr.life
Public WebUI Colab URL: https://flooring-where-severe-parliament.trycloudflare.com
Startup time: 495.8s (import torch: 5.2s, import gradio: 1.2s, import ldm: 1.9s, other imports: 13.4s, list SD models: 3.9s, setup codeformer: 20.5s, load scripts: 123.7s, load SD checkpoint: 154.1s, create ui: 171.7s, gradio launch: 0.2s).
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: ff0680bb (Wed Aug 2 14:00:01 2023)
Starting job task(mobhyct6cd3ry4a)-0...
Unpacking ControlNet 1 base video
Exporting Video Frames to /content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/outputs/img2img-images/Deforum_20230914133053/controlnet_1_inputframes...
*START OF TRACEBACK*
Traceback (most recent call last):
File "/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts/deforum_helpers/run_deforum.py", line 94, in run_deforum
render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root)
File "/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts/deforum_helpers/render.py", line 65, in render_animation
unpack_controlnet_vids(args, anim_args, controlnet_args)
File "/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts/deforum_helpers/deforum_controlnet.py", line 321, in unpack_controlnet_vids
process_controlnet_input_frames(args, anim_args, controlnet_args, vid_path, None, 'inputframes', i)
File "/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts/deforum_helpers/deforum_controlnet.py", line 297, in process_controlnet_input_frames
vid2frames(
File "/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts/deforum_helpers/video_audio_utilities.py", line 65, in vid2frames
raise RuntimeError('Error: extract_to_frame can not be higher than extract_from_frame')
RuntimeError: Error: extract_to_frame can not be higher than extract_from_frame
*END OF TRACEBACK*
User friendly error message:
Error: Error: extract_to_frame can not be higher than extract_from_frame. Check your schedules/ init values please. Also make sure you don't have a backwards slash in any of your PATHs - use / instead of \.
Additional information
Run using the following git repo: https://github.com/camenduru/stable-diffusion-webui-colab/tree/drive
This issue has been closed due to incorrect formatting. Please address the following mistakes and reopen the issue (click on the 'Reopen' button below):
- Make sure the issue title has at least 3 words.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thanks for addressing your formatting mistakes. The issue has been reopened now.
@GeorgeSpiller And did you have Generate input frames switched ON in HybridVideo? That’s a must-do if you’re wanting to use any of the HybridVideo features (if you haven’t already run it once with Generate input frames ON)
This is obviously just a sloppy error message. The logic was correct. The error message was wrong. It said:
if (extract_to_frame <= extract_from_frame) and extract_to_frame != -1:
raise RuntimeError('Error: extract_to_frame cannot be higher than the extract_from_frame')
The original programmer probably meant to say this:
if (extract_to_frame <= extract_from_frame) and extract_to_frame != -1:
raise RuntimeError('Error: extract_from_frame cannot be higher than the extract_to_frame')
They just swapped the to and from in the message.
