If I want to make my own project dataset, how can I get this pkl file.

[xyysname]

real_test\cat\000001-coor.pkl. Hello.What does the content of this pkl file mean? If I want to make my own project dataset, how can I get this pkl file.Can I train the model without this pkl file? What impact will it have?

[hirohitokato]

I also would like to know how to create the XXX-coor.pkl file, too.

[chyphen7]

I would also be interested in knowing how to train our own data to create *-coor.pkl. Thanks.

[LZGMatrix]

@xyysname @hirohitokato @chyphen7 Thanks for your questions. These "*-coor.pkl" coordinates files are the ground-truth coordinates labels to train the rotation head of our CDPN model. They can be created by the 3D object models and the pose annotations provided by the LM/LMO dataset. Concreately, you can simply project the 3D object models to the image plane using the ground-truth pose annotation to create them.

[hirohitokato]

@LZGMatrix Thank you so much for a comment!

Do you use/create the tool for building the "*-coor..pkl" files? If so, I want to use it so that we can train your model with the best performance.

[xyysname]

@LZGMatrix Thanks for your answer, but I want to confirm whether the specific production steps I now know are correct. Now,I don't have depth pictures in my hand, only the ply file (point cloud coordinates) and pose information of the model. Can I directly use the 3D point cloud coordinates of the model, that is, the point cloud coordinates in the ply file, to create the coor in the pkl file? Is the "coor" in the plk file obtained by using the pose matrix to transform the point cloud coordinates in the ply file?

[xyysname]

我现在制作数据集遭遇了制作这个 pkl文件里面the ground-truth coordinates 的困难。这个coor坐标的含义是物体像素点对应的真实的3d坐标，那我怎么能够得到这个真实的3d坐标呢，具体要使用哪些工具呢？我现在手里不能得到深度图片，只有rgb图片和图片对应的pose矩阵以及物体的ply点云文件，那我能用这些东西把那个ground-truth coordinates制作出来吗？

[hirohitokato]

I am trying to create a -coor.pkl file, but I still can't get it to work. I would like to know what tool or scripts you used to convert it.

[petrock99]

I too am interested in setting up my own dataset and am getting hung up by the pkl files. I see that there are .ply & .obj files in root/dataset/lm/models. If would be great if a method to convert one of the 3d model files to a .pkl file were made available. I think that would answer a lot of questions. Thanks.

[petrock99]

@LZGMatrix, correct me if I'm wrong, but I think I was able to reverse engineer the image portion of the pkl file. After some normalizing of the data it appears to be a 2D rendering of the corresponding 3D model with the matching pose.txt rotations applied, as @LZGMatrix mentioned above. Each raw channel value is roughly between -0.05 & 0.05. When normalized to between 0 & 1, to make matplotlib happy, each pixel has a different color of the rainbow.

@LZGMatrix:

• Does the different color per pixel & -0.05 to 0.05 value range matter to the algorithm?
• Is it essentially a unit vector field?
• Should I file a separate 'Issue' to request an official method to generate .pkl files from the .ply/.obj & pose.txt files or would you like to use this one?

I've written a small script to convert the .pkl files to .pngs in case anyone is interested.

import pickle as pkl
from matplotlib import image

def norm(coor):    # Normalize between 0.0 & 1.0
    return ((coor - coor.min()) * 255. / (coor.max() - coor.min())) / 255.

with open("path/to/coor.pkl", "rb") as f:
    image.imsave('coor.png', norm(pkl.load(f)['coor']))

[lululovecode]

我现在制作数据集遭遇了制作这个 pkl文件里面the ground-truth coordinates 的困难。这个coor坐标的含义是物体像素点对应的真实的3d坐标，那我怎么能够得到这个真实的3d坐标呢，具体要使用哪些工具呢？我现在手里不能得到深度图片，只有rgb图片和图片对应的pose矩阵以及物体的ply点云文件，那我能用这些东西把那个ground-truth coordinates制作出来吗？

我也遇到了一样的问题，怎么制作生成pkl文件呢？不然一直无法训练自己的数据集....

[LZGMatrix]

@LZGMatrix Thank you so much for a comment!

Do you use/create the tool for building the "*-coor..pkl" files? If so, I want to use it so that we can train your model with the best performance.

Hi, as said before, these *-coor.pkl are generated by projecting the 3d object model (i.e. *.ply file) using the ground-truth pose annotations. It can be implemented with no more than 100 lines of code.

[LZGMatrix]

@hirohitokato @lululovecode @xyysname @chyphen7 Hi, as said before, these *-coor.pkl are generated by projecting the 3d object model (i.e. *.ply file) using the ground-truth pose annotations. It can be implemented with no more than 100 lines of code. If all people still cannot achive it, maybe in the near future (when I have spare time), I will provide a short technical article to illustrate it. Anyway, it is a very simple and straightforward thing.

[BoBoINVICTUS]

Thanks for your questions. These "*-coor.pkl" coordinates files are the ground-truth coordinates labels to train the rotation head of our CDPN model. They can be created by the 3D object models and the pose annotations provided by the LM/LMO dataset. Concreately, you can simply project the 3D object models to the image plane using the ground-truth pose annotation to create them.

Hello，has this technical article been published?

[LZGMatrix]

@BoBoINVIDICUS

Hi, I will release it in several days, hold on~

[wang-kexian]

大佬你好，请问这个.pkl文件存储的是在当前相机坐标系下的3D坐标吗？可以用深度相机结合相机内参计算的到吗？

[george66s]

@LZGMatrix, just wondering if the report has been released? Thanks for your great effort.

George

[george66s]

@LZGMatrix, I am afraid this is not as straight forward as you said. You probably need a renderer to render the .ply object in order to generate .pkl files, which is the reason that makes it complicated and not straight forward, since render a 3D object is non trivial. Correct me if I am wrong. Someone used OpenGL tools to render. And the code can be hundreds of lines.

[LZGMatrix]

@george66s Hi, if you get bad results such as holes in the coordinates map, you may need to do some interpolation operations in the 3D model or in the projected coordinates map, and it should not cost too many lines~

best

[Yipingggg]

@george66s Hi, if you get bad results such as holes in the coordinates map, you may need to do some interpolation operations in the 3D model or in the projected coordinates map, and it should not cost too many lines~

best

Is is possible that you can release a technical article on the whole thing? Thanks a lot!

[george66s]

The hole issue is exactly why it is non trivial. You may do interpolation when holes are inside the projection. It's quite difficult when holes are on edges of the projection since you don't know if points belong to the object or not. This is why you need a renderer, or you'll have to write a renderer yourself, which is totally not necessary and would probably cost you thousands of lines.

[guan2000910]

现在有代码了吗？我想在另一个数据集上进行训练。

[ToMato-t]

我现在制作数据集遭遇了制作这个 pkl文件，里面the ground-truth coordinates 的困难。这个coor坐标的含义是物体像素点对应的真实的3d坐标，那我怎么能够得到这个真实的3d坐标呢，具体要使用哪些工具呢？我现在手里不能得到深度图片，只有rgb图片和图片对应的pose矩阵以及物体的ply点云文件，那我能用这些东西把那个ground-truth coordinates制作出来吗？

我也遇到了一样的问题，怎么制作生成pkl文件呢？不然一直无法训练自己的数据集....

你好，请问你生成pkl文件成功了吗？？