How to train on custom synthetic dataset?

[mikkeljakobsen]

Hi,

I would like to train MoreFusion on my own synthetic dataset of boxes. I've generated my dataset using BlenderProc and I have ground truth instance masks and 6D poses for the boxes. An example train image from my generated data is shown below: I managed to train Mask R-CNN so it generalizes to real images to some extend. But I need to do full 6D pose estimation.

So my question is: do you think it would be possible to succesfully train MoreFusion on my own custom synthetic data as described above? And what would I need to do in order to adapt the training script to custom data?

Best regards, Mikkel

[wkentaro]

To train the 6D pose estimation model, we need the below information, which probably you can extract from Blender. https://github.com/wkentaro/morefusion/blob/5c1036c19db0a55d24314336b09c01de11284a9f/morefusion/datasets/rgbd_pose_estimation/ycb_video/dataset.py#L73-L80

For the actual training, you need to create your custom dataset class similar to the below. https://github.com/wkentaro/morefusion/blob/5c1036c19db0a55d24314336b09c01de11284a9f/morefusion/datasets/rgbd_pose_estimation/ycb_video/dataset.py#L9

And use it in the train.py script. https://github.com/wkentaro/morefusion/blob/master/examples/ycb_video/singleview_3d/train.py

[alanxuefei]

In ./singleview_3d/train.py, YCBVideoRGBDPoseEstimationDatasetReIndexed is used. YCBVideoRGBDPoseEstimationDatasetReIndexed uses the sample data 1 as below. The above datasets is generated from the sample data 2 as below via MyRealRGBDPoseEstimationDataset The sample data 2 may be collected via data_collection_server.py.

Questions:

• How does these sample data 2 generated?

[hz-ants]

@ mikkeljakobsen @alanxuefei Hi,have you completed the project task of multiple instances of similar objects?Can you share your successful experience,thank you!

[mikkeljakobsen]

@ mikkeljakobsen @alanxuefei Hi,have you completed the project task of multiple instances of similar objects?Can you share your successful experience,thank you!

Sorry, I didn't manage to train MoreFusion on my own data.