Different rendered results

[KelestZ]

Hi, thanks for the great work. I currently have an issue with rendered images (in the last column). As shown, the rendered images are not similar to the input images (the first column). I am just using the default setting. Could you please help me figure out what setting/code I should change in order to get faithful reconstructions? Thanks!

[lkh9908]

Similar question, how do you get an overlayed input image for comparison?

[UestcJay]

Hi, I got similar rendered results, do you now know what went wrong or how to modify the code? Looking forward to your reply!

[1151537695]

Hi, I get the same results and I find only 50 albedo parameters are used, maybe more parameters......

[UestcJay]

Hi, I get the same results and I find only 50 albedo parameters are used, maybe more parameters......

I tried 199 albedo parameters, but it seems to be useless, maybe I am wrong, you can try

[1151537695]

Hi, I get the same results and I find only 50 albedo parameters are used, maybe more parameters......

I tried 199 albedo parameters, but it seems to be useless, maybe I am wrong, you can try

Have you trained your model? If you still get bad results, you can try https://github.com/waps101/AlbedoMM and https://github.com/HavenFeng/photometric_optimization

[425776024]

@KelestZ KelestZ According to the red frame in this picture, you can solve your problem

[qiankl]

@KelestZ KelestZ According to the red frame in this picture, you can solve your problem

your code works. Another problem is how to render an image with a particular yaw/pitch angle. Do you have any idea? I have tried to adjust the 'pose' params, but the texture is added to the wrong place.

[Bornblack]

The output of the original code renders the course model only (last column).

The output of the code from @425776024 renders the course model with a blending texture which combines a unwrapped texture (front face mask region) from the original image and the predicted texture (other part of the flame model) .

Anyone wants to produce the predicted detailed image that calculate the photometric loss could refer these code

  opdict['uv_texture_gt'] = uv_texture_gt
 
  predicted_detail_images = F.grid_sample(opdict['uv_texture'], ops['grid'].detach(), align_corners=False)

  visdict = {
      'inputs': images, 
      'landmarks2d': util.tensor_vis_landmarks(images, landmarks2d),
      'landmarks3d': util.tensor_vis_landmarks(images, landmarks3d),
      'shape_images': shape_images,
      'shape_detail_images': shape_detail_images
  }
  if self.cfg.model.use_tex:
      visdict['rendered_images'] = predicted_detail_images
      # visdict['rendered_images'] = opt['images']
     
  return opdict, visdict

[zoezhu]

For who want the rendered face is as the same postion as input image, the folowing code may be help.

        ops_detail = self.render(verts, trans_verts, opdict['uv_texture'], codedict['light'])

        visdict = {
            'inputs': images, 
            'landmarks2d': util.tensor_vis_landmarks(images, landmarks2d),
            'landmarks3d': util.tensor_vis_landmarks(images, landmarks3d),
            'shape_images': shape_images,
            'shape_detail_images': shape_detail_images
        }
        if self.cfg.model.use_tex:
            # visdict['rendered_images'] = ops['images']  # original
            visdict['rendered_images'] = ops_detail['images']

        return opdict, visdict

[akarshmishra001]

@KelestZ KelestZ According to the red frame in this picture, you can solve your problem

your code works. Another problem is how to render an image with a particular yaw/pitch angle. Do you have any idea? I have tried to adjust the 'pose' params, but the texture is added to the wrong place.

Were you able to make it work ?

[buithikieu11]

@KelestZ KelestZ According to the red frame in this picture, you can solve your problem

do you know how to rotate the image with texture is extracted from source image and predicted texture from FLAME model?I see the demo_teaser.py file but it can only working with the texture is predicted from FLAME model.