any suggestions for training ip adapter with flux

[garychan22]

hi, i am trying to train an ip adapter for flux with a subset of laion-2b (about 3M). the training loss decreases very fast in hundreds of iterations and stays in a small range around 0.3~0.4. after about 60k iterations, the generated images by the model are related to the input prompts but not the input reference images.

are there any suggestions for training ip adapter? i have successfully trained ip adapter for sdxl before and the adopted training dataset is the same, i have also checked: 1) the image features extracted by the pre-trained image encoder which are correct and 2) the new parameters have indeed been trained with fsdp tool.

[bozagina]

Can you leave your email address so we can discuss training issues?

[JoshonSmith]

hi, i am trying to train an ip adapter for flux with a subset of laion-2b (about 3M). the training loss decreases very fast in hundreds of iterations and stays in a small range around 0.3~0.4. after about 60k iterations, the generated images by the model are related to the input prompts but not the input reference images.

are there any suggestions for training ip adapter? i have successfully trained ip adapter for sdxl before and the adopted training dataset is the same, i have also checked: 1) the image features extracted by the pre-trained image encoder which are correct and 2) the new parameters have indeed been trained with fsdp tool.

hi, what's the training code used for flux ipadapter?