comfyui-reactor-node icon indicating copy to clipboard operation
comfyui-reactor-node copied to clipboard
verified publisher icon Gourieff

ONNXRuntimeError : 6 : RUNTIME_EXCEPTION : Exception during initialization:

Open lord-lethris opened this issue 1 year ago • 13 comments

First, confirm

  • [X] I have read the instruction carefully
  • [X] I have searched the existing issues
  • [X] I have updated the extension to the latest version

What happened?

I'm using ZLUDA (switched from DirectML - honestly its miles better), I don't know much about the ins and outs, but I think it kinda emulates CUDA on AMD GPUs, and it works really well...

I've had issues in the past with ReActor not working at all with DirectML and there was a code hack to get it to work. And it was kindly eventually added to to the main trunk. Now I'm having the same issue with ZLUDA.

As a suggestion, can we have the option on the node to force use CPU only?

As a note - ReActor is the only node that doesn't seem to work when running ZLUDA, so I guess it doesn't pull requests for CUDA from the same code and/or routines that all the other nodes use.? (which isn't a moan btw)

Error is:

[ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR ; GPU=0 ; hostname=DESKTOP-GH34TUD ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=178 ; expr=cudnnSetStream(cudnn_handle_, stream);

File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\apps\SD-WebUI\ComfyUI\custom_nodes\comfyui-reactor-node\nodes.py", line 334, in execute
script.process(
File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_faceswap.py", line 91, in process
result = swap_face(
File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_swapper.py", line 230, in swap_face
source_faces = analyze_faces(source_img)
File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_swapper.py", line 147, in analyze_faces
face_analyser = getAnalysisModel(det_size)
File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_swapper.py", line 77, in getAnalysisModel
ANALYSIS_MODEL = insightface.app.FaceAnalysis(
File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\reactor_patcher.py", line 48, in patched_faceanalysis_init
model = model_zoo.get_model(onnx_file, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\insightface\model_zoo\model_zoo.py", line 96, in get_model
model = router.get_model(providers=providers, provider_options=provider_options)
File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\reactor_patcher.py", line 21, in patched_get_model
session = PickableInferenceSession(self.onnx_file, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\insightface\model_zoo\model_zoo.py", line 25, in __init__
super().__init__(model_path, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "D:\apps\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)

Steps to reproduce the problem

Just created a simple workflow

workflow (21)

Sysinfo

OS - Windows 11 Python - 3.10.11 Browser - Edge, also tried Opera GX. CPU - AMD Ryzen 9 5900X 12-Core Processor running at 6 Cores GPU - AMD Radeon RX 6800

Nodes Running (But not necessarily on the workflow):

brightness_contrast_ally clip_text_encode_split gaussian_blur_ally histogram_equalization ComfyUi-NoodleWebcam sharpness_ally DeepCash ComfyUI-VideoHelperSuite saturation_ally imageflip_ally ComfyUI_IPAdapter_plus fcSuite efficiency-nodes-comfyui rembg-comfyui-node ksampler_sdxl SDXLAspectRatio Pseudo_HDR_ally websocket_image_save conditioning_sizing_for_SDXL ComfyUI-Styles_CSV_Loader CLIPTextFromTemplateEncode ComfyUI-sampler-lcm-alternative ComfyUI-StableDiffusion3-API sdxl_prompt_styler ComfyUI_Noise comfyui-optical-flow ComfyUI_ADV_CLIP_emb ComfyUI-Gtsuya-Nodes comfyui_controlnet_aux comfyui-tooling-nodes pose-generator-comfyui-node comfyui-portrait-master ComfyUI_SimpleMath ComfyUI_Cutoff ComfyUI_toyxyz_test_nodes ComfyUI_JPS-Nodes ComfyUI-post-processing-nodes ComfyUI_TiledIPAdapter ComfyUI_essentials comfyui-ultralytics-yolo comfy_clip_blip_node ComfyUI_TiledKSampler ComfyUi_NNLatentUpscale ComfyUI-OpenPose-Editor SD-Advanced-Noise ComfyUI-WD14-Tagger ComfyUI-TGu-utils ComfyUI_experiments comfy-plasma ComfyUI-QualityOfLifeSuit_Omar92 ComfyQR-scanning-nodes ComfyUI_FizzNodes comfyui-inpaint-nodes ComfyMath ComfyUI_NestedNodeBuilder comfyui-animatediff images-grid-comfy-plugin ComfyUI_Load_Image_With_Metadata failfast-comfyui-extensions ComfyUI_UltimateSDUpscale ComfyUI-Advanced-ControlNet ComfyQR tri3d-comfyui-nodes ComfyUI-Marigold ComfyUI-AnimateDiff-Evolved rgthree-comfy ComfyUI-NegiTools ComfyUI_Comfyroll_CustomNodes comfy_mtb comfyui-prompt-control comfyui-prompt-reader-node facerestore_cf comfyui-reactor-node ComfyUI_tinyterraNodes ComfyUI-Hangover-Moondream SeargeSDXL Comfyui_segformer_b2_clothes ComfyUI-Frame-Interpolation ComfyUI-KJNodes comfyui-dynamicprompts ComfyUI-APISR clipseg ComfyUI-Custom-Scripts ComfyUI-Manager ComfyUI-Inspire-Pack ComfyUI-Allor Jovimetrix ComfyUI-Portrait-Maker was-node-suite-comfyui comfyui-mixlab-nodes ComfyUI-Crystools ComfyUI-Impact-Pack ComfyUI_Custom_Nodes_AlekPet ComfyUI-DiffusersStableCascade comfyui-art-venture

Relevant console log

got prompt
[rgthree] Using rgthree's optimized recursive execution.
100%|█████████████████████████████████████████████████████████████| 15/15 [00:04<00:00,  3.59it/s]
[ReActor] 17:00:35 - STATUS - Working: source face index [0], target face index [0]
[ReActor] 17:00:35 - STATUS - Analyzing Source Image...
2024-05-22 17:00:35.8014171 [E:onnxruntime:, inference_session.cc:1981 onnxruntime::InferenceSession::Initialize::<lambda_b88fb06a047eb3fba81e04c489d52b7d>::operator ()] Exception during initialization: D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR ; GPU=0 ; hostname=DESKTOP-GH34TUD ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=178 ; expr=cudnnSetStream(cudnn_handle_, stream);


!!! Exception during processing!!! [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR ; GPU=0 ; hostname=DESKTOP-GH34TUD ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=178 ; expr=cudnnSetStream(cudnn_handle_, stream);


Traceback (most recent call last):
  File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "D:\apps\SD-WebUI\ComfyUI\custom_nodes\comfyui-reactor-node\nodes.py", line 334, in execute
    script.process(
  File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_faceswap.py", line 91, in process
    result = swap_face(
  File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_swapper.py", line 230, in swap_face
    source_faces = analyze_faces(source_img)
  File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_swapper.py", line 147, in analyze_faces
    face_analyser = getAnalysisModel(det_size)
  File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\scripts\reactor_swapper.py", line 77, in getAnalysisModel
    ANALYSIS_MODEL = insightface.app.FaceAnalysis(
  File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\reactor_patcher.py", line 48, in patched_faceanalysis_init
    model = model_zoo.get_model(onnx_file, **kwargs)
  File "D:\apps\Python\Python310\lib\site-packages\insightface\model_zoo\model_zoo.py", line 96, in get_model
    model = router.get_model(providers=providers, provider_options=provider_options)
  File "E:\ASSETS\AI-TOOLS\COMFY-DATA\custom_nodes\comfyui-reactor-node\reactor_patcher.py", line 21, in patched_get_model
    session = PickableInferenceSession(self.onnx_file, **kwargs)
  File "D:\apps\Python\Python310\lib\site-packages\insightface\model_zoo\model_zoo.py", line 25, in __init__
    super().__init__(model_path, **kwargs)
  File "D:\apps\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "D:\apps\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR ; GPU=0 ; hostname=DESKTOP-GH34TUD ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=178 ; expr=cudnnSetStream(cudnn_handle_, stream);



Prompt executed in 5.27 seconds

Additional information

No response

lord-lethris avatar May 22 '24 16:05 lord-lethris