UV map to SMPL model

[Amran2k16]

Hi,

Once i have the UV maps for the images after using create_uv_maps.py , how do i construct a SMPL model out of it?

[liwenssss]

hi, @Amran2k16 , have you done it, I want to do it too

[Amran2k16]

Yeah, you need to use the resample function that can be found in the uv_map_generator python file. This file can be found in the data_utils folder and takes a uv map as input argument

[liwenssss]

thanks for your response ,I will try later~

[liwenssss]

hi, @Amran2k16 , I use create_UV_maps.py to generate some uv_map, and I use the resample function like this:

    file_prefix = 'template'
    generator = UV_Map_Generator(
        UV_height=256,
        UV_pickle=file_prefix+'.pickle'
    )
    uv_map = cv2.imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0000.jpg")
    resample_mesh = generator.resample(uv_map)

    device = torch.device('cuda')

    model = SMPLModel(
        device=device,
        model_path='./model_lsp.pkl',
        data_type=torch.float32
    )

    model.write_obj(resample_mesh, './test.obj')

the uv_map is like this: but it seem I got the wrong res, the generate obj file show like this: any advice can you give?

[Amran2k16]

Can you try and use smpl_fbx_temate instead of the radvani ones and show me what results you get?

[Lotayou]

Hi @liwenssss I think you made a mistake here:

uv_map = cv2.imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0000.jpg")
# uv_map = uv_map.astype(np.float) / 255.0
resample_mesh = generator.resample(uv_map)

Maybe you should convert the uv_map to float between [0,1] first. Also, I would not suggest storing uv maps as jpg, since the lossy compression would change the coordinates. It's better to save them as lossless image format like png or bmp.

[liwenssss]

@Amran2k16 are you it convenient to send me your generate smpl_fbx_temate file. Actually， I fail to generate it so that I use the radvani's. Maybe it is the reasaon. @Lotayou , thanks for your advice , but it seems not the reason. I do the following change:

   uv_map = imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0001.png")
    uv_map = uv_map.astype(np.float) / 255.0
    resample_mesh = generator.resample(uv_map)

    device = torch.device('cuda')

    model = SMPLModel(
        device=device,
        model_path='./model_lsp.pkl',
        data_type=torch.float32
    )

    model.write_obj(resample_mesh, './test.obj')

and get the res like this:

[Amran2k16]

@liwenssss Hi, what changes when you stop dividing your UV_map by 255? So uv_map = uv_map.astype(np.float) this instead of uv_map = uv_map.astype(np.float) / 255.0.

[liwenssss]

@Amran2k16 Hi, I tried. nothing changed.

uv_map = imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0001.png")
    uv_map = uv_map.astype(np.float)
    print(uv_map.min(), uv_map.max())
    resample_mesh = generator.resample(uv_map)

    device = torch.device('cuda')

    model = SMPLModel(
        device=device,
        model_path='./model_lsp.pkl',
        data_type=torch.float32
    )

    model.write_obj(resample_mesh, './test.obj')

and got print info: Find cached pickle file... 0.0 254.0 (7670, 2) 7670 Tensor J_regressor shape: torch.Size([24, 6890]) Tensor joint_regressor shape: torch.Size([6890, 14]) Tensor weights shape: torch.Size([6890, 24]) Tensor posedirs shape: torch.Size([6890, 3, 207]) Tensor v_template shape: torch.Size([6890, 3]) Tensor shapedirs shape: torch.Size([6890, 3, 10])

by the way , imread is from skimage.io

[Amran2k16]

@liwenssss Sorry, not entirely sure what the issue is in that case. Maybe it might be the template issue, but i doubt it. Unfortunately I don't think i can send you the generate smpl_fbx_temate file due to the SMPL license.

As for generating it, these are the steps i took

1. Import SMPL FBX file into blender (You will need to get this file from SMPL website)
2. Export it as an obj file, using the triangulation options that can be selected (this way you will not need to use triangulation.py method, I myself did not use this but discovered it much later)

[Lotayou]

@liwenssss This is quite odd. Seems your resampled mesh still looks like a person, so it's minor mistake... How about sending me an working example and I'll try it out on my machine?

[liwenssss]

thanks your advice, I just successfully generate the smpl_fbx_template.obj. I find the reason is that I am not familiar with blender. I will try later to solve this and if I solve it I will tell you

------------------ 原始邮件 ------------------ 发件人: "Amran2k16"[email protected]; 发送时间: 2019年8月23日(星期五) 晚上7:11 收件人: "Lotayou/densebody_pytorch"[email protected]; 抄送: "李文盛"[email protected]; "Mention"[email protected]; 主题: Re: [Lotayou/densebody_pytorch] UV map to SMPL model (#41)

@liwenssss Sorry, not entirely sure what the issue is in that case. Maybe it might be the template issue, but i doubt it. Unfortunately I don't think i can send you the generate smpl_fbx_temate file due to the SMPL license.

As for generating it, these are the steps i took

Import SMPL FBX file into blender (You will need to get this file from SMPL website)

Export it as an obj file, using the triangulation options that can be selected (this way you will not need to use triangulation.py method, I myself did not use this but discovered it much later)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[liwenssss]

I just think maybe I do something wrong I dont notice. I will try to do it from the beginning process except the data generate process. I am not sure what file do you want.

------------------ 原始邮件 ------------------ 发件人: "Lingbo Yang"[email protected]; 发送时间: 2019年8月23日(星期五) 晚上7:34 收件人: "Lotayou/densebody_pytorch"[email protected]; 抄送: "李文盛"[email protected]; "Mention"[email protected]; 主题: Re: [Lotayou/densebody_pytorch] UV map to SMPL model (#41)

@liwenssss This is quite odd. Seems your resampled mesh still looks like a person, so it's minor mistake... How about sending me an working example and I'll try it out on my machine?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Lotayou]

okay, good luck

[liwenssss]

hi, @Amran2k16 ,have you met the following phenomenon when you generate SMPL model: as we can see some bulge around the left hand and the right feet. if you met it and solve it ,bother to tell me how? or any advice can you give me ?

[Amran2k16]

Hi, I believe this is an issue of UV MAP size. I am guessing this was reshaped from a 256 * 256 image. Instead try to do it with a 512 * 512 image size. Also did you find that triangulation.py was not a necessary step if it was done in blender already?

[tszhang97]

this is my resample result.Some tips:

1. when normalize to [0,1], the result will be different from the groud truth
2. may not use dilate, this will cause those bugle
3. using bilinear interpolation rather than nearest integer. And find the nearest nonzero point if 4 nearing points all equal 0

[Amran2k16]

@willie1997 How did you manage to create UV maps without normalizing to [0,1]? And which part of the process did you use bilinear interpolation in exactly?

[tszhang97]

normalization is a problem which I haven't figured out. If you want to remain the correct propotion, my solution is saving vmin and vmax. vt*256 will calculate a result like [150.22, 223.15], you can use the value of its nearest four points to interpolation. @Amran2k16

[tszhang97]

Good news! I finally fix the bug and resample a very smooth smpl model. The tip is remove the vts equal zero in v_to_vt list and only use the good verts .

[gaizixuan0128]

@willie1997 Hello！Could you share your resample code？I have followed your tips，but still couldn‘t solve the bulge problem.

[electronicliujiang]

@liwenssss hello!I don't have smpl_fbx_template file.so,I use the radvani's.this is my result,do you know how cai i do or where to find smpl_fbx_template file.I found this file is gone on the official website.

[PerhapsThat]

@liwenssss hello!I don't have smpl_fbx_template file.so,I use the radvani's.this is my result,do you know how cai i do or where to find smpl_fbx_template file.I found this file is gone on the official website.

hi,have u solved the problem? I have encontered the same bug as yours