train text diffuser on customized dataset

[lwb2099]

Describe Model I am using textdiffuser: Hi, I am training textdiffuser using my customized dataset, and I wonder how to build segmentation mask information. It seems that there is no code for generating segmentation mask from raw image. Can you provide the code fro preparation? or should we use UNet for generating segmentation mask.

[JingyeChen]

https://github.com/microsoft/unilm/blob/master/textdiffuser/inference.py#L434C33-L434C33

[simajiucai]

https://github.com/microsoft/unilm/blob/master/textdiffuser/inference.py#L434C33-L434C33 in the code, the segmentation mask is generated by a user-defined prompt. but how can we build the segmentation mask from raw image? @JingyeChen