MagicDance
MagicDance copied to clipboard
Boese0601
[ICML 2024] MagicPose(also known as MagicDance): Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion
hi folks @Boese0601 et al., thx for ur contribution! i found the net design shares quite a similarity with the AA work, e.g., the essential "Appearance Control Model" vs. ReferenceNet....
Issue solved :) _Originally posted by @Boese0601 in https://github.com/Boese0601/MagicDance/issues/18#issuecomment-1975676943_ I've been troubled by this problem for a long time and would really appreciate your help.
When running the inference_tiktok_dataset.sh script, pose_control_model (i.e., ControlNet) is running unnecessarily (in line 1112 ~ 1114 in cldm.py) for computing model_uncod. When uc==False, ControlNetReferenceOnly --> ControlNet (pose_control_model) --> ControlledUnetModelAttnPose (OK)...
Tiktok naturally didn't include any training code.... https://github.com/bytedance/X-Portrait it would be good to be able to retrofit this code base with their updates... they also give up on the body...
Dear Author, Thank you for sharing source code! But I find that when using ```CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 torchrun --master_port 10000 --nproc_per_node 8 train_tiktok.py ```, every gpu reads the same data in every...
Thank you for your work. I would like to ask, in the first stage, is the training loss calculated as the loss between the generated image and the image in...
``` (magicpose) root@autodl-container-98884cbd0f-2e382a5b:~/autodl-tmp# bash scripts/inference_any_image_pose.sh TORCH_VERSION=1 FP16_DTYPE=torch.float16 Customize text prompt: None Namespace(model_config='model_lib/ControlNet/models/cldm_v15_reference_only_pose.yaml', reinit_hint_block=False, image_size=64, empty_text_prob=0.1, sd_locked=True, only_mid_control=False, finetune_all=False, finetune_imagecond_unet=False, control_type=['body+hand+face'], control_dropout=0.0, depth_bg_threshold=0.0, inpaint_unet=False, blank_mask_prob=0.0, mask_densepose=0.0, control_mode='controlnet_important', wonoise=True, mask_bg=False, img_bin_limit='all', num_workers=1,...
Dear author, I tried to train your model on one 4090 gpu. I reduce the batch_size to 1 to fit the gpu memory, but it gives me this error during...
About the initialization of Stable Diffusion weights, you provided the control_sd15_ini.ckpt model, which I understand is a weight extracted from SD1.5 using ControlNet. The weight file you provided is 5.7GB....
