comfyanonymous
Could not run 'torchvision::nms' with arguments from the 'CUDA' backend
I got stuck when I ran FaceDetailerPipe and the system reported the error: "Could not run 'torchvision::nms' with arguments from the 'CUDA' backend" even though I installed CUDA as required including:
- Cuda 12.1
- cuDNN 9
- Pytorch 2.2.0+cu121
- Torchvision 0.17.0+cu121
I have uninstalled pytorch and reinstalled it many times including change setting device = "cpu" but it still doesn't work. Please help me fix it
Here is the full error log:
FETCH DATA from: D:\Pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Manager\extension-node-map.json got prompt model_type EPS adm 0 Using xformers attention in VAE Working with z of shape (1, 4, 32, 32) = 4096 dimensions. Using xformers attention in VAE missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'} left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids']) Requested to load SD1ClipModel Loading 1 new model D:\Pinokio\api\comfyui.git\app\comfy\ldm\modules\attention.py:325: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.) out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False) Requested to load BaseModel Loading 1 new model 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 5.77it/s] Requested to load AutoencoderKL Loading 1 new model Loads SAM model: D:\Pinokio\api\comfyui.git\app\models\sams\sam_vit_b_01ec64.pth (device:AUTO)
0: 640x640 (no detections), 45.0ms Speed: 12.0ms preprocess, 45.0ms inference, 24.0ms postprocess per image at shape (1, 3, 640, 640) Prompt executed in 12.60 seconds got prompt 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.22it/s]
ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "D:\Pinokio\api\comfyui.git\app\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "D:\Pinokio\api\comfyui.git\app\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "D:\Pinokio\api\comfyui.git\app\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "D:\Pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_pack.py", line 1225, in doit enhanced_img, cropped_enhanced, cropped_enhanced_alpha, mask, cnet_pil_list = FaceDetailer.enhance_face( File "D:\Pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_pack.py", line 446, in enhance_face segs = bbox_detector.detect(image, bbox_threshold, bbox_dilation, bbox_crop_factor, drop_size, detailer_hook=detailer_hook) File "D:\Pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Impact-Pack\impact_subpack\impact\subcore.py", line 106, in detect detected_results = inference_bbox(self.bbox_model, core.tensor2pil(image), threshold) File "D:\Pinokio\api\comfyui.git\app\custom_nodes\ComfyUI-Impact-Pack\impact_subpack\impact\subcore.py", line 33, in inference_bbox pred = model(image, conf=confidence, device=device) File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\ultralytics\engine\model.py", line 155, in call return self.predict(source, stream, **kwargs) File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\ultralytics\engine\model.py", line 406, in predict return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream) File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\ultralytics\engine\predictor.py", line 204, in call return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\torch\utils_contextlib.py", line 35, in generator_context response = gen.send(None) File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\ultralytics\engine\predictor.py", line 290, in stream_inference self.results = self.postprocess(preds, im, im0s) File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\ultralytics\models\yolo\detect\predict.py", line 25, in postprocess preds = ops.non_max_suppression( File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\ultralytics\utils\ops.py", line 277, in non_max_suppression i = torchvision.ops.nms(boxes, scores, iou_thres) # NMS File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\torchvision\ops\boxes.py", line 41, in nms return torch.ops.torchvision.nms(boxes, scores, iou_threshold) File "C:\Users\Admin\pinokio\api\comfyui.git\app\env\lib\site-packages\torch_ops.py", line 755, in call return self._op(*args, **(kwargs or {})) NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using cu stom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::nms' is only available for these backends: [CPU, Meta, QuantizedCPU, BackendSelec t, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, Autograd Meta, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, BatchedNestedTensor, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PreDispatch, PythonDispatcher].
CPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\cpu\nms_kernel.cpp:112 [kernel] Meta: registered at /dev/null:440 [kernel] QuantizedCPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\quantized\cpu\qnms_kernel.cpp:124 [kernel] BackendSelect: fallthrough registered at ..\aten\src\ATen\core\BackendSelectFallbackKernel.cpp:3 [backend fallback] Python: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:154 [backend fallback] FuncTorchDynamicLayerBackMode: registered at ..\aten\src\ATen\functorch\DynamicLayer.cpp:498 [backend fallback] Functionalize: registered at ..\aten\src\ATen\FunctionalizeFallbackKernel.cpp:324 [backend fallback] Named: registered at ..\aten\src\ATen\core\NamedRegistrations.cpp:7 [backend fallback] Conjugate: registered at ..\aten\src\ATen\ConjugateFallback.cpp:17 [backend fallback] Negative: registered at ..\aten\src\ATen\native\NegateFallback.cpp:19 [backend fallback] ZeroTensor: registered at ..\aten\src\ATen\ZeroTensorFallback.cpp:86 [backend fallback] ADInplaceOrView: fallthrough registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:86 [backend fallback] AutogradOther: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:53 [backend fallback] AutogradCPU: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:57 [backend fallback] AutogradCUDA: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:65 [backend fallback] AutogradXLA: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:69 [backend fallback] AutogradMPS: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:77 [backend fallback] AutogradXPU: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:61 [backend fallback] AutogradHPU: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:90 [backend fallback] AutogradLazy: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:73 [backend fallback] AutogradMeta: registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:81 [backend fallback] Tracer: registered at ..\torch\csrc\autograd\TraceTypeManual.cpp:297 [backend fallback] AutocastCPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autocast\nms_kernel.cpp:34 [kernel] AutocastCUDA: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autocast\nms_kernel.cpp:27 [kernel] FuncTorchBatched: registered at ..\aten\src\ATen\functorch\LegacyBatchingRegistrations.cpp:720 [backend fallback] BatchedNestedTensor: registered at ..\aten\src\ATen\functorch\LegacyBatchingRegistrations.cpp:746 [backend fallback] FuncTorchVmapMode: fallthrough registered at ..\aten\src\ATen\functorch\VmapModeRegistrations.cpp:28 [backend fallback] Batched: registered at ..\aten\src\ATen\LegacyBatchingRegistrations.cpp:1075 [backend fallback] VmapMode: fallthrough registered at ..\aten\src\ATen\VmapModeRegistrations.cpp:33 [backend fallback] FuncTorchGradWrapper: registered at ..\aten\src\ATen\functorch\TensorWrapper.cpp:203 [backend fallback] PythonTLSSnapshot: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:162 [backend fallback] FuncTorchDynamicLayerFrontMode: registered at ..\aten\src\ATen\functorch\DynamicLayer.cpp:494 [backend fallback] PreDispatch: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:166 [backend fallback] PythonDispatcher: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:158 [backend fallback]
Full file here comfyui.log
https://github.com/ltdrdata/ComfyUI-Impact-Pack/issues/405 https://github.com/ultralytics/ultralytics/issues/5059
There is version mismatch between torch packages.
https://github.com/ltdrdata/ComfyUI-Impact-Pack/issues/405
https://github.com/ultralytics/ultralytics/issues/5059
There is version mismatch between torch packages.
I saw the information you said that faceDetailer stops working from now on, right?
I saw the information you said that faceDetailer stops working from now on, right?
I meant that you need to fix the version mismatch of installed torch related packages. The easiest way is to reconstruct the installation environment that has not been messed up by installing different packages.
I don't know if this gonna work for you, but i got the same error somes days ago. I resolved that by creating a .bat in the same folder has = "Comfyui, python_embeded, update" folders with the line :
.\python_embeded\python.exe -m pip install --force-reinstall torchvision --index-url https://download.pytorch.org/whl/cu121
And everythings is working again
I don't know if this gonna work for you, but i got the same error somes days ago. I resolved that by creating a .bat in the same folder has = "Comfyui, python_embeded, update" folders with the line :
.\python_embeded\python.exe -m pip install --force-reinstall torchvision --index-url https://download.pytorch.org/whl/cu121And everythings is working again
thanks, I solved by reinstall and its gonna work. I'm using pikono so I can not find python_embeded folder to create a .bat file
I saw the information you said that faceDetailer stops working from now on, right?
I meant that you need to fix the version mismatch of installed torch related packages. The easiest way is to reconstruct the installation environment that has not been messed up by installing different packages.
thanks bro, I solved by reading clealy & slowly your thread mentioned. So, I reinstall comfyui and impact pake only then it's worked
I don't know if this gonna work for you, but i got the same error somes days ago. I resolved that by creating a .bat in the same folder has = "Comfyui, python_embeded, update" folders with the line :
.\python_embeded\python.exe -m pip install --force-reinstall torchvision --index-url https://download.pytorch.org/whl/cu121And everythings is working again
Did this work by just creating an empty .bat file and running it or going into the directory and copy and pasting the command to download it?
I don't know if this gonna work for you, but i got the same error somes days ago. I resolved that by creating a .bat in the same folder has = "Comfyui, python_embeded, update" folders with the line :
.\python_embeded\python.exe -m pip install --force-reinstall torchvision --index-url https://download.pytorch.org/whl/cu121And everythings is working again
Thanks, it works for me.
