[Bug]: Deforum FILM/RIFE interpolation doesn't work with AMD/ZLUDA installations

[x5nder]

Have you read the latest version of the FAQ?

• [X] I have visited the FAQ page right now and my issue is not present there

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the Deforum extension?

• [X] I have Deforum updated to the lastest version and I still have the issue.

What happened?

Running Automatic1111 v1.9.3-amd with --use-zluda (on an RX 7900 GRE).

FILM and RIFE interpolation crash (regardless of being invoked from AnimateDiff or Deforum tab). See output in the console logs below.

Steps to reproduce the problem

1. Go to Deforum tab, Output, Interpolate existing Video/ Images
2. Load any video file
3. Select RIFE or FILM
4. Set Interp X to any number (eg 2) then click Interpolate Video

What should have happened/how would you fix it?

No response

Torch version

torch: 2.2.1+cu118

On which platform are you launching the webui with the extension?

Local PC setup (Windows)

Deforum settings

20240426120615_settings.txt

Webui core settings

ui-config.json

Console logs

WARNING: ZLUDA works best with SD.Next. Please consider migrating to SD.Next.
Using ZLUDA in C:\StableDiffusion\ZLUDA-windows-amd64
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.9.3-amd
Commit hash: 6130ef9e7a194bb0a212d12a697347153c4f31e2
All models for DeOldify are already downloaded.
All requirements for the DeOldify extension are already installed.
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Launching Web UI with arguments: --use-zluda --listen --enable-insecure-extension-access
C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
ONNX: selected=CUDAExecutionProvider, available=['CPUExecutionProvider']
[-] ADetailer initialized. version: 24.4.2, num models: 10
CivitAI Browser+: Aria2 RPC started
ControlNet preprocessor location: C:\StableDiffusion\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\downloads
2024-04-26 12:25:35,335 - ControlNet - INFO - ControlNet v1.1.445
2024-04-26 12:25:35,409 - ControlNet - INFO - ControlNet v1.1.445
Loading weights [84d76a0328] from C:\StableDiffusion\stable-diffusion-webui-directml\models\Stable-diffusion\epicrealism_naturalSinRC1VAE.safetensors
Creating model from config: C:\StableDiffusion\stable-diffusion-webui-directml\configs\v1-inference.yaml
Applying attention optimization: Doggettx... done.
Model loaded in 2.0s (create model: 0.3s, apply weights to model: 1.0s, load textual inversion embeddings: 0.2s, calculate empty prompt: 0.4s).
2024-04-26 12:25:38,229 - ControlNet - INFO - ControlNet UI callback registered.
*Deforum ControlNet support: enabled*
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 15.1s (prepare environment: 9.8s, initialize shared: 1.1s, load scripts: 2.1s, create ui: 3.0s, gradio launch: 4.2s).
got a request to *frame interpolate* an existing video.
Trying to extract frames from video with input FPS of 33.333333333333336. Please wait patiently.
Extracted 290 frames from video in 0.21 seconds!
Total frames to FILM-interpolate: 290. Total frame-pairs: 289.
FILM progress:   0%|▋                                                                                                                                                                                      | 1/289 [00:00<03:57,  1.21it/s]
Traceback (most recent call last):
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\gradio_funcs.py", line 117, in upload_vid_to_interpolate
    process_interp_vid_upload_logic(file, engine, x_am, sl_enabled, sl_am, keep_imgs, f_location, f_crf, f_preset, in_vid_fps, f_models_path, file.orig_name)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\frame_interpolation.py", line 82, in process_interp_vid_upload_logic
    process_video_interpolation(frame_interpolation_engine=engine, frame_interpolation_x_amount=x_am, frame_interpolation_slow_mo_enabled = sl_enabled,frame_interpolation_slow_mo_amount=sl_am, orig_vid_fps=in_vid_fps, deforum_models_path=f_models_path, real_audio_track=audio_file_to_pass, raw_output_imgs_path=outdir, img_batch_id=None, ffmpeg_location=f_location, ffmpeg_crf=f_crf, ffmpeg_preset=f_preset, keep_interp_imgs=keep_imgs, orig_vid_name=folder_name, resolution=resolution)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\frame_interpolation.py", line 117, in process_video_interpolation
    return prepare_film_inference(deforum_models_path=deforum_models_path, x_am=frame_interpolation_x_amount, sl_enabled=frame_interpolation_slow_mo_enabled, sl_am=frame_interpolation_slow_mo_amount, keep_imgs=keep_interp_imgs, raw_output_imgs_path=raw_output_imgs_path, img_batch_id=img_batch_id, f_location=ffmpeg_location, f_crf=ffmpeg_crf, f_preset=ffmpeg_preset, fps=fps, audio_track=real_audio_track, orig_vid_name=orig_vid_name, is_random_pics_run=is_random_pics_run, srt_path=srt_path)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\frame_interpolation.py", line 161, in prepare_film_inference
    run_film_interp_infer(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\src\film_interpolation\film_inference.py", line 90, in run_film_interp_infer
    prediction = model(x0, x1, dt)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
RuntimeError: nvrtc: error: invalid value for --gpu-architecture (-arch)

nvrtc compilation failed:

#define NAN __int_as_float(0x7fffffff)
#define POS_INFINITY __int_as_float(0x7f800000)
#define NEG_INFINITY __int_as_float(0xff800000)


template<typename T>
__device__ T maximum(T a, T b) {
  return isnan(a) ? a : (a > b ? a : b);
}

template<typename T>
__device__ T minimum(T a, T b) {
  return isnan(a) ? a : (a < b ? a : b);
}


#define __HALF_TO_US(var) *(reinterpret_cast<unsigned short *>(&(var)))
#define __HALF_TO_CUS(var) *(reinterpret_cast<const unsigned short *>(&(var)))
#if defined(__cplusplus)
  struct __align__(2) __half {
    __host__ __device__ __half() { }

  protected:
    unsigned short __x;
  };

  /* All intrinsic functions are only available to nvcc compilers */
  #if defined(__CUDACC__)
    /* Definitions of intrinsics */
    __device__ __half __float2half(const float f) {
      __half val;
      asm("{  cvt.rn.f16.f32 %0, %1;}\n" : "=h"(__HALF_TO_US(val)) : "f"(f));
      return val;
    }

    __device__ float __half2float(const __half h) {
      float val;
      asm("{  cvt.f32.f16 %0, %1;}\n" : "=f"(val) : "h"(__HALF_TO_CUS(h)));
      return val;
    }

  #endif /* defined(__CUDACC__) */
#endif /* defined(__cplusplus) */
#undef __HALF_TO_US
#undef __HALF_TO_CUS

typedef __half half;

extern "C" __global__
void fused_unsqueeze_unsqueeze_unsqueeze_div(half* tv_, half* tv__, half* output_1) {
{
  float v = __half2float(tv_[(long long)(threadIdx.x) + 512ll * (long long)(blockIdx.x)]);
  float v_1 = __half2float(tv__[((long long)(threadIdx.x) + 512ll * (long long)(blockIdx.x)) / 27648ll]);
  output_1[(long long)(threadIdx.x) + 512ll * (long long)(blockIdx.x)] = __float2half(v / v_1);
}
}

got a request to *frame interpolate* an existing video.
Trying to extract frames from video with input FPS of 33.333333333333336. Please wait patiently.
Extracted 290 frames from video in 0.21 seconds!
RIFE46.pkl model successfully loaded into memory
Interpolation progress (it's OK if it finishes before 100%):
  0%|                                                                                                                                                                                                              | 0/290 [00:00<?, ?it/s]Traceback (most recent call last):
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\gradio_funcs.py", line 117, in upload_vid_to_interpolate
    process_interp_vid_upload_logic(file, engine, x_am, sl_enabled, sl_am, keep_imgs, f_location, f_crf, f_preset, in_vid_fps, f_models_path, file.orig_name)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\frame_interpolation.py", line 82, in process_interp_vid_upload_logic
    process_video_interpolation(frame_interpolation_engine=engine, frame_interpolation_x_amount=x_am, frame_interpolation_slow_mo_enabled = sl_enabled,frame_interpolation_slow_mo_amount=sl_am, orig_vid_fps=in_vid_fps, deforum_models_path=f_models_path, real_audio_track=audio_file_to_pass, raw_output_imgs_path=outdir, img_batch_id=None, ffmpeg_location=f_location, ffmpeg_crf=f_crf, ffmpeg_preset=f_preset, keep_interp_imgs=keep_imgs, orig_vid_name=folder_name, resolution=resolution)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\frame_interpolation.py", line 115, in process_video_interpolation
    return run_rife_new_video_infer(interp_x_amount=frame_interpolation_x_amount, slow_mo_enabled = frame_interpolation_slow_mo_enabled, slow_mo_x_amount=frame_interpolation_slow_mo_amount, model=actual_model_folder_name, fps=fps, deforum_models_path=deforum_models_path, audio_track=real_audio_track, raw_output_imgs_path=raw_output_imgs_path, img_batch_id=img_batch_id, ffmpeg_location=ffmpeg_location, ffmpeg_crf=ffmpeg_crf, ffmpeg_preset=ffmpeg_preset, keep_imgs=keep_interp_imgs, orig_vid_name=orig_vid_name, UHD=UHD, srt_path=srt_path)
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\src\rife\inference_video.py", line 154, in run_rife_new_video_infer
    ssim = ssim_matlab(I0_small[:, :3], I1_small[:, :3])
  File "C:\StableDiffusion\stable-diffusion-webui-directml\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\src\rife\model\pytorch_msssim\__init__.py", line 107, in ssim_matlab
    mu1 = F.conv3d(F.pad(img1, (5, 5, 5, 5, 5, 5), mode='replicate'), window, padding=padd, groups=1)
RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR

Additional information

No response