DiffusionCLIP icon indicating copy to clipboard operation
DiffusionCLIP copied to clipboard

Replication Issue (Figure 26)

Open harveymannering opened this issue 5 months ago • 0 comments

I am having problem replicating figure 26 (https://arxiv.org/pdf/2110.02711). Specifically, I am trying to edit the celeb1.png image to be "angry", "with makeup", and "zombie". Here are the results from the paper:

However, when running this repos code I get different results. Here are the images I got for "angry", "with makeup", and "zombie":

These images don't seem quite as good and I am not sure why.

Could this be an issue with the hyperparameters? I used the hyperparameters in table 5 for each of these images. For inference I used 200 inversion steps and 40 test steps. For more info on the parameters I used here are the commands I ran for training and for inference of the "angry" model:

# Training
python main.py --clip_finetune \
    --config celeba.yml \
    --exp ./runs/test \
    --edit_attr angry \
    --do_train 1 --do_test 1 \
    --n_train_img 50 \
    --n_test_img 10 \
    --n_iter 5  --t_0 500 \
    --n_inv_step 40 --n_train_step 6 \ 
    --n_test_step 40 \
    --lr_clip_finetune 8e-6 \
    --id_loss_w 0.3 --l1_loss_w 0.3 \
    --model_path celeba_hq.ckpt \
    --clip_model_name ViT-B-16.pt

# Inference
python main.py --edit_one_image  \
    --img_path imgs/celeb1.png \
    --config celeba.yml \
    --exp ./runs/test \
    --t_0 500 \
    --n_inv_step 200 --n_test_step 40 \
    --n_iter 1 \
    --model_path checkpoint/angry.pth

Any suggestions on what hyperparameters I could change to get the same results are figure 26? Or are there any ideas as to what else may be causing this discrepancy between the figure 26 and my results?

harveymannering avatar Sep 17 '24 13:09 harveymannering