Controlling face on the output image with ControlNet.

[diruuu]

Is it possible to use ControlNet architecture to create a model that works like a model produced by LoRA training?

For example: In Stable Diffusion, I have 2 inputs: text prompt and an image of someone's face. I want this hypothetical ControlNet model to use the exact someone's face on the output image without having need to use LoRA model or something. Is it possible to create this kind of ControlNet model?

[offchan42]

This model is the closest from what I've seen on the internet: https://www.reddit.com/r/StableDiffusion/comments/1281iva/new_controlnet_face_model/

It's used to mostly control facial expression though. If you want EXACT face copy then better use ControlNet models that deal with line edges (e.g. canny edge). If you want similar face, then the default ControlNet architecture cannot do the use case you want.

[jiqizaisikao]

这是我自己模型训练的结果，类似于Adobe的生成填充，微信最新的<换影CP>小程序，根据一张照片，一个模版，进行一键换头，一键换脸型。 不过需要不小的算力进行训练，我可能没有那么多资源进行训练优化了，希望可以找到朋友进行合作研究和开发

[jiqizaisikao]

[jiqizaisikao]

[geroldmeisinger]

use ControlNet 1.1 open pose https://github.com/lllyasviel/ControlNet-v1-1-nightly#controlnet-11-openpose or SDXL open pose https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0