[Fix text_model confinement][Unexpected key(s) in state_dict]

[tomguluson92]

Fix Unexpected key(s) in state_dict: "text_model.embeddings.position_ids". problem aroused by loading text_model.

[kohya-ss]

Thank you for this! But I'd like to keep strict=True to find unexpected errors. If some error occurs in loading the state dict, I will add a conversion code for it. Could you please tell me the location (url) of the checkpoint which causes the error?

[tomguluson92]

Thank you for this! But I'd like to keep strict=True to find unexpected errors. If some error occurs in loading the state dict, I will add a conversion code for it. Could you please tell me the location (url) of the checkpoint which causes the error?

Thanks, This is only for info text_model, which is not used in real training. The base model I found not compatible is civitai-awportrait

[kohya-ss]

I've updated dev branch to support the model, and will merge into main sooner. May I close this PR?

[vladmandic]

btw, this is also happening if using transformers any version higher than 4.30.2.

[8-unlimit]

When I upgraded transformers==4.36.2 this error disappeared