PhotoMaker icon indicating copy to clipboard operation
PhotoMaker copied to clipboard

目前 ComfyUI 是不是不支持 PhotoMaker V2 ?

Open O-O1024 opened this issue 1 year ago • 4 comments

加载 PhotoMaker V2 权值出错了

!!! Exception during processing!!! Error(s) in loading state_dict for PhotoMakerIDEncoder:
        Unexpected key(s) in state_dict: "qformer_perceiver.perceiver_resampler.layers.0.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.0.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.0.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.0.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.0.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.0.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.0.1.3.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.1.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.1.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.1.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.1.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.1.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.1.1.3.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.2.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.2.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.2.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.2.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.2.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.2.1.3.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.3.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.3.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.3.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.3.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.3.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.3.1.3.weight", "qformer_perceiver.perceiver_resampler.norm_out.bias", "qformer_perceiver.perceiver_resampler.norm_out.weight", "qformer_perceiver.perceiver_resampler.proj_in.bias", "qformer_perceiver.perceiver_resampler.proj_in.weight", "qformer_perceiver.perceiver_resampler.proj_out.bias", "qformer_perceiver.perceiver_resampler.proj_out.weight", "qformer_perceiver.token_norm.bias", "qformer_perceiver.token_norm.weight", "qformer_perceiver.token_proj.0.bias", "qformer_perceiver.token_proj.0.weight", "qformer_perceiver.token_proj.2.bias", "qformer_perceiver.token_proj.2.weight". 
Traceback (most recent call last):
  File "/root/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "/root/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "/root/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "/root/ComfyUI/comfy_extras/nodes_photomaker.py", line 134, in load_photomaker_model
    photomaker_model.load_state_dict(data)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for PhotoMakerIDEncoder:
        Unexpected key(s) in state_dict: "qformer_perceiver.perceiver_resampler.layers.0.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.0.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.0.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.0.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.0.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.0.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.0.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.0.1.3.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.1.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.1.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.1.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.1.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.1.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.1.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.1.1.3.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.2.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.2.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.2.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.2.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.2.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.2.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.2.1.3.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.norm1.bias", "qformer_perceiver.perceiver_resampler.layers.3.0.norm1.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.norm2.bias", "qformer_perceiver.perceiver_resampler.layers.3.0.norm2.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.to_kv.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.to_out.weight", "qformer_perceiver.perceiver_resampler.layers.3.0.to_q.weight", "qformer_perceiver.perceiver_resampler.layers.3.1.0.bias", "qformer_perceiver.perceiver_resampler.layers.3.1.0.weight", "qformer_perceiver.perceiver_resampler.layers.3.1.1.weight", "qformer_perceiver.perceiver_resampler.layers.3.1.3.weight", "qformer_perceiver.perceiver_resampler.norm_out.bias", "qformer_perceiver.perceiver_resampler.norm_out.weight", "qformer_perceiver.perceiver_resampler.proj_in.bias", "qformer_perceiver.perceiver_resampler.proj_in.weight", "qformer_perceiver.perceiver_resampler.proj_out.bias", "qformer_perceiver.perceiver_resampler.proj_out.weight", "qformer_perceiver.token_norm.bias", "qformer_perceiver.token_norm.weight", "qformer_perceiver.token_proj.0.bias", "qformer_perceiver.token_proj.0.weight", "qformer_perceiver.token_proj.2.bias", "qformer_perceiver.token_proj.2.weight".

O-O1024 avatar Jul 25 '24 21:07 O-O1024

We currently don't know which ComfyUI node can support photomaker v2.

Paper99 avatar Jul 26 '24 08:07 Paper99

I'm confused v2 was released on Hugging Space, but I see from a note that you deleted it because it was v1?

WeirdScienceX avatar Jul 26 '24 22:07 WeirdScienceX

I've moved the V2 checkpoint to a new model repository: https://huggingface.co/TencentARC/PhotoMaker-V2 Besides, PhotoMaker V2 ComfyUI nodes can be found in:

  1. https://github.com/shiimizu/ComfyUI-PhotoMaker-Plus
  2. https://github.com/edwios/ComfyUI-PhotoMakerV2-ZHO/tree/main

Paper99 avatar Jul 27 '24 15:07 Paper99

also can try this: https://github.com/zhangp365/ComfyUI_photomakerV2_native

zhangp365 avatar Oct 31 '24 08:10 zhangp365