ComfyUI icon indicating copy to clipboard operation
ComfyUI copied to clipboard
Published 1 day ago • verified publisher icon comfyanonymous

Inserting `comfy/` path into `sys.path` and having `comfy/types.py` will break all modules requiring `types` built-in library

Open w568w opened this issue 1 year ago • 1 comments

Your question

Note: I encountered this issue while using a third-party custom node, so I suppose I should post it as a "User Support"? This issue actually affects not only third-party nodes, and I have found the root cause when writing this issue!

When using the Model Compile+ node from https://github.com/cubiq/ComfyUI_essentials, I encountered the following error:

Traceback (most recent call last):
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_worker/__main__.py", line 2, in <module>
    import argparse
  File "/usr/lib/python3.12/argparse.py", line 89, in <module>
    import re as _re
  File "/usr/lib/python3.12/re/__init__.py", line 124, in <module>
    import enum
  File "/usr/lib/python3.12/enum.py", line 3, in <module>
    from types import MappingProxyType, DynamicClassAttribute
  File "/home/w568w/Packages/ComfyUI/comfy/types.py", line 1, in <module>
    import torch
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/__init__.py", line 16, in <module>
    import platform
  File "/usr/lib/python3.12/platform.py", line 120, in <module>
    import functools
  File "/usr/lib/python3.12/functools.py", line 22, in <module>
    from types import GenericAlias
ImportError: cannot import name 'GenericAlias' from partially initialized module 'types' (most likely due to a circular import) (/home/w568w/Packages/ComfyUI/comfy/types.py)

Reproduce steps

  1. Install https://github.com/cubiq/ComfyUI_essentials in a fresh ComfyUI;
  2. Add a Model Compile+ node after Load Checkpoint node to compile the model;
  3. Click Queue;
  4. See the errors.

Logs

Log Details 

got prompt
model_type EPS
model weight dtype torch.float16, manual cast: torch.float32
Using pytorch attention in VAE
Using pytorch attention in VAE
Traceback (most recent call last):
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_worker/__main__.py", line 2, in <module>
    import argparse
  File "/usr/lib/python3.12/argparse.py", line 89, in <module>
    import re as _re
  File "/usr/lib/python3.12/re/__init__.py", line 124, in <module>
    import enum
  File "/usr/lib/python3.12/enum.py", line 3, in <module>
    from types import MappingProxyType, DynamicClassAttribute
  File "/home/w568w/Packages/ComfyUI/comfy/types.py", line 1, in <module>
    import torch
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/__init__.py", line 16, in <module>
    import platform
  File "/usr/lib/python3.12/platform.py", line 120, in <module>
    import functools
  File "/usr/lib/python3.12/functools.py", line 22, in <module>
    from types import GenericAlias
ImportError: cannot import name 'GenericAlias' from partially initialized module 'types' (most likely due to a circular import) (/home/w568w/Packages/ComfyUI/comfy/types.py)
W0804 16:22:21.170000 138623546558144 torch/_inductor/compile_worker/subproc_pool.py:126] SubprocPool unclean exit
Requested to load SDXLClipModel
Loading 1 new model
Warning torch.load doesn't support weights_only on this pytorch version, loading unsafely.
Requested to load SDXL
Loading 1 new model
loading in lowvram mode 2233.27451171875
  0%|                                                                                                                                             | 0/8 [00:00<?, ?it/s]W0804 16:23:31.897000 138625066993344 torch/_inductor/utils.py:977] [0/0] Not enough SMs to use max_autotune_gemm mode
W0804 16:23:33.176000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] r0 is not in var_ranges, defaulting to unknown range.
W0804 16:23:34.708000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] r0 is not in var_ranges, defaulting to unknown range.
W0804 16:23:42.565000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] q1 is not in var_ranges, defaulting to unknown range.
W0804 16:23:42.613000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] z1 is not in var_ranges, defaulting to unknown range.
W0804 16:23:42.854000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] q0 is not in var_ranges, defaulting to unknown range.
W0804 16:23:42.884000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] z0 is not in var_ranges, defaulting to unknown range.
W0804 16:23:43.485000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] q1 is not in var_ranges, defaulting to unknown range.
W0804 16:23:43.533000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] z1 is not in var_ranges, defaulting to unknown range.
W0804 16:23:44.418000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] q0 is not in var_ranges, defaulting to unknown range.
W0804 16:23:44.448000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] z0 is not in var_ranges, defaulting to unknown range.
AUTOTUNE addmm(12800x640, 12800x320, 320x640)
  addmm 3.5390 ms 100.0%
  bias_addmm 4.0702 ms 86.9%
SingleProcess AUTOTUNE benchmarking takes 0.4681 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(12800x640, 12800x640, 640x640)
  bias_addmm 6.7669 ms 100.0%
  addmm 6.8293 ms 99.1%
SingleProcess AUTOTUNE benchmarking takes 0.3363 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(3200x1280, 3200x640, 640x1280)
  bias_addmm 3.2925 ms 100.0%
  addmm 3.4079 ms 96.6%
SingleProcess AUTOTUNE benchmarking takes 0.2886 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(3200x1280, 3200x1280, 1280x1280)
  bias_addmm 6.6879 ms 100.0%
  addmm 6.7030 ms 99.8%
SingleProcess AUTOTUNE benchmarking takes 0.3299 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(3200x1280, 3200x2560, 2560x1280)
  bias_addmm 13.3394 ms 100.0%
  addmm 13.4564 ms 99.1%
SingleProcess AUTOTUNE benchmarking takes 0.3805 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(3200x1280, 3200x1920, 1920x1280)
  bias_addmm 10.0157 ms 100.0%
  addmm 10.2072 ms 98.1%
SingleProcess AUTOTUNE benchmarking takes 0.3606 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(12800x640, 12800x1920, 1920x640)
  bias_addmm 19.9884 ms 100.0%
  addmm 20.2038 ms 98.9%
SingleProcess AUTOTUNE benchmarking takes 0.4702 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(12800x640, 12800x1280, 1280x640)
  bias_addmm 13.4216 ms 100.0%
  addmm 13.4551 ms 99.8%
SingleProcess AUTOTUNE benchmarking takes 0.3840 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(12800x640, 12800x960, 960x640)
  addmm 10.1294 ms 100.0%
  bias_addmm 10.1313 ms 100.0%
SingleProcess AUTOTUNE benchmarking takes 0.3632 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(51200x320, 51200x960, 960x320)
  bias_addmm 20.0518 ms 100.0%
  addmm 20.7451 ms 96.7%
SingleProcess AUTOTUNE benchmarking takes 0.4772 seconds and 0.0000 seconds precompiling
AUTOTUNE addmm(51200x320, 51200x640, 640x320)
  bias_addmm 13.5918 ms 100.0%
  addmm 13.8859 ms 97.9%
SingleProcess AUTOTUNE benchmarking takes 0.3919 seconds and 0.0000 seconds precompiling
W0804 16:23:59.248000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] x1 is not in var_ranges, defaulting to unknown range.
W0804 16:23:59.439000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] x1 is not in var_ranges, defaulting to unknown range.
W0804 16:23:59.555000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] y0 is not in var_ranges, defaulting to unknown range.
W0804 16:23:59.885000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] x1 is not in var_ranges, defaulting to unknown range.
W0804 16:24:00.657000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] x1 is not in var_ranges, defaulting to unknown range.
W0804 16:24:00.773000 138625066993344 torch/fx/experimental/symbolic_shapes.py:4449] [0/0] y0 is not in var_ranges, defaulting to unknown range.
  0%|                                                                                                                                             | 0/8 [01:31<?, ?it/s]
!!! Exception during processing!!! backend='inductor' raised:
BrokenPipeError: [Errno 32] Broken pipe

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

Traceback (most recent call last):
  File "/home/w568w/Packages/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/nodes.py", line 1382, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/nodes.py", line 1352, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 22, in informative_sample
    raise e
  File "/home/w568w/Packages/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 9, in informative_sample
    return original_sample(*args, **kwargs)  # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/sample.py", line 43, in sample
    samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 829, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 729, in sample
    return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 716, in sample
    output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 695, in inner_sample
    samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 600, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/k_diffusion/sampling.py", line 160, in sample_euler_ancestral
    denoised = model(x, sigmas[i] * s_in, **extra_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 299, in __call__
    out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 682, in __call__
    return self.predict_noise(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 685, in predict_noise
    return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 279, in sampling_function
    out = calc_cond_batch(model, conds, x, timestep, model_options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/samplers.py", line 228, in calc_cond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/comfy/model_base.py", line 123, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 1116, in __call__
    return self._torchdynamo_orig_callable(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 472, in __call__
    return _compile(
           ^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_utils_internal.py", line 84, in wrapper_function
    return StrobelightCompileTimeProfiler.profile_compile_time(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner
    out_code = transform_code_object(code, transform)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/bytecode_transformation.py", line 1185, in transform_code_object
    transformations(instructions, code_options)
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform
    tracer.run()
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2451, in run
    super().run()
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
          ^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2642, in RETURN_VALUE
    self._return(inst)
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/symbolic_convert.py", line 2627, in _return
    self.output.compile_subgraph(
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/output_graph.py", line 1123, in compile_subgraph
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/output_graph.py", line 1318, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/output_graph.py", line 1409, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/output_graph.py", line 1390, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.example_inputs())
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/repro/after_dynamo.py", line 129, in __call__
    compiled_gm = compiler_fn(gm, example_inputs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/repro/after_dynamo.py", line 129, in __call__
    compiled_gm = compiler_fn(gm, example_inputs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/__init__.py", line 1951, in __call__
    return compile_fx(model_, inputs_, config_patches=self.config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1261, in compile_fx
    return compile_fx(
           ^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1505, in compile_fx
    return aot_autograd(
           ^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/backends/common.py", line 69, in __call__
    cg = aot_module_simplified(gm, example_inputs, **self.kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_functorch/aot_autograd.py", line 954, in aot_module_simplified
    compiled_fn, _ = create_aot_dispatcher_function(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_functorch/aot_autograd.py", line 687, in create_aot_dispatcher_function
    compiled_fn, fw_metadata = compiler_fn(
                               ^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 168, in aot_dispatch_base
    compiled_fw = compiler(fw_module, updated_flat_args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1410, in fw_compiler_base
    return inner_compile(
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/repro/after_aot.py", line 84, in debug_wrapper
    inner_compiled_fn = compiler_fn(gm, example_inputs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/debug.py", line 304, in inner
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 527, in compile_fx_inner
    compiled_graph = fx_codegen_and_compile(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 831, in fx_codegen_and_compile
    compiled_fn = graph.compile_to_fn()
                  ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/graph.py", line 1749, in compile_to_fn
    return self.compile_to_module().call
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/graph.py", line 1699, in compile_to_module
    mod = PyCodeCache.load_by_key_path(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 3062, in load_by_key_path
    mod = _reload_python_module(key, path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/runtime/compile_tasks.py", line 45, in _reload_python_module
    exec(code, mod.__dict__, mod.__dict__)
  File "/tmp/torchinductor_w568w/hv/chvk57ceeovwscpx7gfxuul7q2nyxall6soursq4zt7juw7e4iyr.py", line 33, in <module>
    triton_poi_fused__to_copy_0 = async_compile.triton('triton_', '''
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/async_compile.py", line 177, in triton
    self.process_pool().submit(
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 117, in submit
    _send_msg(self.write_pipe, job_id, job_data)
  File "/home/w568w/Packages/ComfyUI/.venv/lib/python3.12/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 53, in _send_msg
    write_pipe.flush()
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
BrokenPipeError: [Errno 32] Broken pipe

Other

This looks very strange: One of torch's worker processes called import argparse, which eventually called import types, and then tried to import comfy/types.py?

I checked where torch starts the compile worker and found the following code:

self.process = subprocess.Popen(
            cmd,
            env={
                **os.environ,
                # We need to set the PYTHONPATH so the subprocess can find torch.
                "PYTHONPATH": os.pathsep.join(sys.path),
# ...

Here, torch passes sys.path of the current interpreter to its child worker process.

However, in nodes.py, comfy/ is inserted as the first item in sys.path:

https://github.com/comfyanonymous/ComfyUI/blob/56f3c660bf79769bbfa003c0e4152dfb50feadc5/nodes.py#L19

And unfortunately, there is a file named comfy/types.py, which have the same name as Python built-in library types.

Therefore, any subsequent import types will try to import comfy/types.py, which breaks many built-in modules, such as argparse.

Possible Solution

To reorder the import priority, I change sys.path.insert(0, ...) to sys.path.append(...) in nodes.py. The error disappears after that.

w568w avatar Aug 04 '24 08:08 w568w

This should be fixed by #3564.

huchenlei avatar Aug 04 '24 14:08 huchenlei

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.

github-actions[bot] avatar Mar 13 '25 11:03 github-actions[bot]

Is this fixed? I see that the PR is closed without merging.

w568w avatar Mar 13 '25 13:03 w568w

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.

github-actions[bot] avatar Sep 03 '25 11:09 github-actions[bot]

Has been fixed since 9d720187f181bc5a74b7bd36b8ad6276b1336000.

w568w avatar Sep 03 '25 11:09 w568w