Fooocus icon indicating copy to clipboard operation
Fooocus copied to clipboard

Faceswap and loras

Open techalexx opened this issue 1 year ago • 6 comments

  1. faceswap should be the last stage and not the first, otherwise the connected loras will distort the portrait resemblance. Actually this is a bug!
  2. need a combined weight for image prompt for all connected photos at the same time
  3. store image prompt presets for next jobs

techalexx avatar Jan 20 '24 08:01 techalexx

Hey @techalexx,

  1. can't verify if this is intended or not, maybe @lllyasviel is able to help here
  2. feel free to provide a PR for this
  3. please elaborate, the prompt/styles aren't deleted after clicking generated (between jobs). Do you refer to closing and opening the tab of Fooocus? If yes, this issue (enhancement) already exists, see https://github.com/lllyasviel/Fooocus/issues/1622, but is in contrast with the concept of presets.

mashb1t avatar Jan 21 '24 00:01 mashb1t

  1. this is important if you don't want to change the balance between several photos but need to reduce the overall weight. 03

  2. that's not exactly what I meant. since gradio can not to save presets, this can be done using config.txt : Expand settings. Add settings for images, for random and seed and etc. Also add the ability to set multiple paths in one key.

  3. new feature 04

techalexx avatar Jan 21 '24 09:01 techalexx

  1. faceswap should be the last stage and not the first, otherwise the connected loras will distort the portrait resemblance. Actually this is a bug!
  2. need a combined weight for image prompt for all connected photos at the same time
  3. store image prompt presets for next jobs

Consider if you run the faceswap last then you cannot apply styles. The faceswap will usually keep whatever original style comes from the source. If you are trying to swap a face from a photo into say, a cartoon, the face will look mostly as a photo merged into a cartoon.

raymondgp avatar Jan 29 '24 19:01 raymondgp

  1. faceswap should be the last stage and not the first, otherwise the connected loras will distort the portrait resemblance. Actually this is a bug!
  2. need a combined weight for image prompt for all connected photos at the same time
  3. store image prompt presets for next jobs

Consider if you run the faceswap last then you cannot apply styles. The faceswap will usually keep whatever original style comes from the source. If you are trying to swap a face from a photo into say, a cartoon, the face will look mostly as a photo merged into a cartoon.

I think as a general principle you want face swap to prioritize fidelity over blending with the style of the surrounding image. If the likeness is high fidelity, blending it with the surrounding image in subsequent passes is much easier than dealing with a lower fidelity image due to whatever happens after swapping. You can always do another pass on the swapped image and apply a style afterwards.

But if the output image doesn't look like the individual being swapped, then the image is unusable no matter how beautifully blended it is.

That said, I'm not sure that the implementation of face swap fooocus uses works this way, where face swap is a distinct step. I get the sense that the likeness is being applied at every step throughout the generation. So I'm not sure this is a bug either.

skylerblack2 avatar Jan 31 '24 00:01 skylerblack2

Even if this is not a bug, it is a flaw. This cannot be used

techalexx avatar Jan 31 '24 08:01 techalexx

I think InstantID solve this problem with Lora not sure if photobooth too

barepixels avatar Feb 15 '24 20:02 barepixels