stable-diffusion-webui-depthmap-script

stable-diffusion-webui-depthmap-script copied to clipboard

So Im trying to process a video to create a depth map out of it and also stereoscopic images, it finishes doing all the steps but when it gets to saving the output it crashes with the following error.. I'm using the latest A1111 and your latest plugin.

Here is the output

100%|███████████████████████████████████████████| 34/34 [09:14<00:00, 16.32s/it]
Computing output(s) done.
All done.

Saving generated frames as video outputs
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/VideoClip.py", line 262, in write_videofile
    codec = extensions_dict[ext]['codec'][0]
KeyError: 'codec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 79, in frames_to_video
    clip.write_videofile(os.path.join(path, f"{name}.{v_format}"), codec=codec, bitrate=br)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 54, in requires_duration
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 135, in use_clip_fps_by_default
    return f(clip, *new_a, **new_kw)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 22, in convert_masks_to_RGB
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/VideoClip.py", line 264, in write_videofile
    raise ValueError("MoviePy couldn't find the codec associated "
ValueError: MoviePy couldn't find the codec associated with the filename. Provide the 'codec' parameter in write_videofile.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/VideoClip.py", line 262, in write_videofile
    codec = extensions_dict[ext]['codec'][0]
KeyError: 'codec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 79, in frames_to_video
    clip.write_videofile(os.path.join(path, f"{name}.{v_format}"), codec=codec, bitrate=br)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 54, in requires_duration
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 135, in use_clip_fps_by_default
    return f(clip, *new_a, **new_kw)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 22, in convert_masks_to_RGB
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/VideoClip.py", line 264, in write_videofile
    raise ValueError("MoviePy couldn't find the codec associated "
ValueError: MoviePy couldn't find the codec associated with the filename. Provide the 'codec' parameter in write_videofile.
Moviepy - Building video /storage/auto1111/depthmap-0-red-cyan-anaglyph_video.mp4.
Moviepy - Writing video /storage/auto1111/depthmap-0-red-cyan-anaglyph_video.mp4

Traceback (most recent call last):
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 79, in frames_to_video
    clip.write_videofile(os.path.join(path, f"{name}.{v_format}"), codec=codec, bitrate=br)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 54, in requires_duration
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 135, in use_clip_fps_by_default
    return f(clip, *new_a, **new_kw)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 22, in convert_masks_to_RGB
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/VideoClip.py", line 300, in write_videofile
    ffmpeg_write_video(self, filename, fps, codec,
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/io/ffmpeg_writer.py", line 213, in ffmpeg_write_video
    with FFMPEG_VideoWriter(filename, clip.size, fps, codec = codec,
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/io/ffmpeg_writer.py", line 88, in __init__
    '-r', '%.02f' % fps,
TypeError: must be real number, not NoneType
Moviepy - Building video /storage/auto1111/depthmap-0-red-cyan-anaglyph_video.webm.
Moviepy - Writing video /storage/auto1111/depthmap-0-red-cyan-anaglyph_video.webm

Traceback (most recent call last):
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 79, in frames_to_video
    clip.write_videofile(os.path.join(path, f"{name}.{v_format}"), codec=codec, bitrate=br)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 54, in requires_duration
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 135, in use_clip_fps_by_default
    return f(clip, *new_a, **new_kw)
  File "/usr/local/lib/python3.9/dist-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/decorators.py", line 22, in convert_masks_to_RGB
    return f(clip, *a, **k)
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/VideoClip.py", line 300, in write_videofile
    ffmpeg_write_video(self, filename, fps, codec,
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/io/ffmpeg_writer.py", line 213, in ffmpeg_write_video
    with FFMPEG_VideoWriter(filename, clip.size, fps, codec = codec,
  File "/usr/local/lib/python3.9/dist-packages/moviepy/video/io/ffmpeg_writer.py", line 88, in __init__
    '-r', '%.02f' % fps,
TypeError: must be real number, not NoneType
Traceback (most recent call last):
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/common_ui.py", line 451, in run_generate
    ret = video_mode.gen_video(
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 156, in gen_video
    frames_to_video(fps, imgs, outpath, f"depthmap-{backbone.get_next_sequence_number(outpath, basename)}-{basename}",
  File "/notebooks/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/src/video_mode.py", line 85, in frames_to_video
    raise Exception('Saving the video failed!')
Exception: Saving the video failed!

This is quite weird...

I see you are using system-wide installed moviepy and python 3.9. You could try creating and using a venv - this may solve the issue.

This is quite weird...

I see you are using system-wide installed moviepy and python 3.9. You could try creating and using a venv - this may solve the issue.

hmm, yea its a virtual environment in a online cloud hosting for gpus that I use. Why would this fail I guess is the first question, i tried to debug it but didnt have time.. It works locally for me but takes for ever on my system to process, also what is moviepy being used for.. I do have fffmpeg setup also, which avi output can sorta suck, in fact I have had to use handbreak to convert it to a mp4 since even my vlc player wont play the avi.. but thats another issue.. btw thanks for all the hard work

This looks like it's a ffmpeg issue because it's missing a codec.

maybe try ffmpeg -version If it's lower than 4 your might need to update ffmpeg.

if it doesn't work you might need to find ffmpeg.

If memory serves, mp4 does significantly reduce the quality because it's compressed 8bit vs the 16bit video. 16bit video is supported in blender. But it's good to know about usage, because maybe having 8bit video is useful too.

This looks like it's a ffmpeg issue because it's missing a codec.

maybe try ffmpeg -version If it's lower than 4 your might need to update ffmpeg.

if it doesn't work you might need to find ffmpeg.

If memory serves, mp4 does significantly reduce the quality because it's compressed 8bit vs the 16bit video. 16bit video is supported in blender. But it's good to know about usage, because maybe having 8bit video is useful too.

hmm, I used deforum a lot and it outputs via ffmpeg, I also use ffmpeg for my local stuff.

Here is the cloud servers info

built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared```

My local command for stuff images to video is something along these lines



```ffmpeg -f image2 -framerate 30 -i 20230925172719_%09d.png -c:v libx264 -preset veryslow -crf 12 -color_primaries 1 -color_trc 1 -colorspace 1 -pix_fmt yuv420p Part10.mp4```

Okay so that looks good. To check for the particular codec (I think it's ffv1 but I might need to double check). ffmpeg -v quiet -codecs | grep '.*DE.*ffv1' the output should be something like DEV..S ffv1 FFmpeg video codec #1

ffmpeg -v quiet -codecs | grep '.*DE.*ffv1'

root@n2k4gvb2gy:/storage/auto1111/Deforum_sara/inputframes# ffmpeg -v quiet -codecs | grep '.*DE.*ffv1' DEV..S ffv1 FFmpeg video codec #1

I was a way for a few days. any thoughts on this?