ali-vilab
dataset issue
I would like to confirm the construction of the training data, such as TryOn. According to the training instructions, six fields are required:
"edit_image": represents the input image for the editing task. If it is not an editing task but a reference generation, this field can be left empty. "edit_mask": represents the input image mask for the editing task, used to specify the editing area. If it is not an editing task but rather for reference generation, this field can be left empty. "ref_image": represents the input image for the reference image generation task; if it is a pure editing task, this field can be left empty. "target_image": represents the generated target image and cannot be empty. "prompt": represents the prompt for the generation task. "data_type": represents the type of data, which can be 'portrait', 'subject', or 'local'. This field is not used in training phase.
The six fields corresponding to the dataset I have prepared are as follows:
"edit_image":
"edit_mask":
"ref_image":
"target_image":
"prompt":
Change the cloth in {image} to the one in {image1}
"data_type":
subject
May I ask if the construction of this dataset is correct
When training for 200 steps, the loss increases and the training fails to obtain the image:
edit_image0：
modify_image0：
edit_mask0
target_image
result:
What problem is causing this? We look forward to your reply