Multiple materials on an object do not get reversed

[jc211]

This was referenced in another issue but it was then closed without a resolution. If an object has multiple materials, the materials are not reversed after the instance material is applied. The issue is in the replace_collection code. I don't how how blender associates materials to different vertices but that doesn't seem to be in the material itself. So even though bpycv is restoring the list of materials, the data about where they are applied is lost.

[DIYer22]

Thanks for reporting BUG. The way of rendering instance annotation is replacing all materials to self luminous material with unique color. I plan to change the way of rendering instance annotation, refrence to https://www.youtube.com/watch?v=xeprI8hJAH8 . Here is a temporary solution: If an object has multiple materials, the temporary solution is to reload objec after rendering the instance annotation.

If you plan to fix this bug, pull requests are welcome.😊

[jc211]

Thanks for the link. That's a great way of doing it. Also makes the rendering faster since it doesn't have to do two passes anymore.

[jc211]

How did you plan on integrating this? Its easy enough to create those nodes, but how do you deal with the nodes the user already has in place?

[DIYer22]

Sorry, I am not familiar with Compositing Nodes and have not figured out how to implement it properly currently.

[jc211]

Here is code that does what the youtube instructions say. I am not sure how it would work in the library yet.

import bpy 

view_layer = bpy.context.view_layer
view_layer.use_pass_z = True
view_layer.use_pass_object_index = True

scene = bpy.context.scene
scene.render.use_compositing = True
scene.use_nodes = True

tree = scene.node_tree
links = tree.links

render_layers = tree.nodes.new(type="CompositorNodeRLayers")
render_layers.location = (-0, 0)

div_block = tree.nodes.new(type="CompositorNodeMath")
render_layers.location = (-300, 200)
div_block.operation = 'DIVIDE'
div_block.inputs[1].default_value = 65535
link = links.new(render_layers.outputs["IndexOB"], div_block.inputs[0])

composite_block = tree.nodes.new(type="CompositorNodeComposite")
composite_block.location = (400, 200)
link = links.new(render_layers.outputs["Image"], composite_block.inputs["Image"])

fileoutput_block = tree.nodes.new(type="CompositorNodeOutputFile")
fileoutput_block.location = (400, 0)
fileoutput_block.file_slots.new("Instance")
fileoutput_block.file_slots["Instance"].format.color_mode = "BW"
fileoutput_block.file_slots["Instance"].format.color_depth = "16"
link = links.new(render_layers.outputs["Image"], fileoutput_block.inputs["Image"])
link = links.new(div_block.outputs[0], fileoutput_block.inputs["Instance"])

[jch-q]

This is also an issue for me as I use many objects with multiple materials.

For the temporary solution, how do you reload the object after rendering the instance annotation in code?

The first image renders fine but from then on the materials do not reset properly.

This is my script:

import cv2
import bpy
import bpycv
import random
import numpy as np
import math

for i in range(3):

    object_1 = bpy.data.objects.get("multi_material_object_1")

    x = 100*random.uniform(-2, 2)
    y = 100*random.uniform(-2, 2)
    z = 100*random.uniform(0, 2)

    rx = random.uniform(0, 2*math.pi)
    ry = random.uniform(0, 2*math.pi)
    rz = random.uniform(0, 2*math.pi)

    object_1.matrix_world.translation = (x,y,z)
    object_1.rotation_euler = (rx,ry,rz)

    object_1["inst_id"] = 1001

    object_2 = bpy.data.objects.get("multi_material_object_2")

    x = 100*random.uniform(-2, 2)
    y = 100*random.uniform(-2, 2)
    z = 100*random.uniform(0, 2)

    rx = random.uniform(0, 2*math.pi)
    ry = random.uniform(0, 2*math.pi)
    rz = random.uniform(0, 2*math.pi)

    object_2.matrix_world.translation = (x,y,z)
    object_2.rotation_euler = (rx,ry,rz)

    object_2["inst_id"] = 2001

    # render image, instance annoatation and depth in one line code
    result = bpycv.render_data()

    # visualization instance mask, RGB, depth for human
    cv2.imwrite(str(i)+"-"+"demo-vis(inst_rgb_depth).jpg", result.vis()[..., ::-1])

[DIYer22]

@jch-q

object_1 = bpy.data.objects.get("multi_material_object_1")
# change to
object_1 = bpycv.load_obj("path/to/3d_file")

[jch-q]

This unfortunately doesn't work for me because I use procedural textures and I want to adjust them each iteration of the loop. Is there another work-around to solve this issue?

[wish2023]

I lose my material too after an iteration. When I use bpycv.load_obj("path/to/3d_file") I get KeyError: 'fbx' (or KeyError: 'blend')

@DIYer22 How do I use bpycv.load_obj()? This is my simplified implementation:

car_path = "vehicles/white_car.fbx"

bpy.ops.import_scene.fbx(filepath=car_path)
# bpycv.load_obj(car_path)

for i in range(2):
    render_name = f"synthetics{i}.png"

    result = bpycv.render_data()
    cv2.imwrite(render_name, result["image"][..., ::-1])

[DIYer22]

@wish2023 Right now, bpycv.load_obj() not support load .fbx, Just using bpy.ops.import_scene.fbx(filepath=car_path)