realfusion
realfusion copied to clipboard
janus problem
Thank you for your work! I found that there was a janus problem in the result of the "teddy bear" example, because the textural inversion was over-fitted to its front view, resulting in not producing the correct rear view, which could lead to janus problems. Will there be such a phenomenon in the official result?
https://github.com/lukemelas/realfusion/assets/78358378/43055208-c1f3-4a98-913e-e08d7524cb8e
My command is as follows:
export MODEL_NAME="/home/litaiqing/.cache/huggingface/hub/models--runwayml--stable-diffusion-v1-5/snapshots/aa9ba505e1973ae5cd05f5aedd345178f52f8e6a"
export DATA_DIR="/media/ssd_1/litaiqing/realfusion-main/examples/natural-images/teddy_bear_1"
export OUTPUT_DIR="/media/ssd_1/litaiqing/realfusion-main/examples/natural-images/teddy_bear_1"
CUDA_VISIBLE_DEVICES=7 python textual_inversion.py \
--pretrained_model_name_or_path=$MODEL_NAME \
--train_data_dir=$DATA_DIR \
--learnable_property="object" \
--placeholder_token="_teddy_bear_" \
--initializer_token="teddy " \
--resolution=512 \
--train_batch_size=1 \
--gradient_accumulation_steps=4 \
--max_train_steps=3000 \
--learning_rate=5.0e-04 --scale_lr \
--lr_scheduler="constant" \
--lr_warmup_steps=0 \
--output_dir=$OUTPUT_DIR \
--use_augmentations
export DATA_DIR=/media/ssd_1/litaiqing/realfusion-main/examples/natural-images/teddy_bear_1
CUDA_VISIBLE_DEVICES=7 python main.py --O \
--image_path $DATA_DIR/rgba.png \
--learned_embeds_path $DATA_DIR/learned_embeds.bin \
--text "a _teddy_bear_" \
--pretrained_model_name_or_path "/home/litaiqing/.cache/huggingface/hub/models--runwayml--stable-diffusion-v1-5/snapshots/aa9ba505e1973ae5cd05f5aedd345178f52f8e6a"