Cannot get the benchmark result using KPConv on S3DIS dataset.

[maosuli]

Checklist

• [X] I have searched for similar issues.
• [X] I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

Using RandLA-Net on S3DIS with the default randlanet_s3dis.yml can get the benchmark result. But I cannot get the ideal mIoU result when using KPConv with the default config file on the same S3DIS dataset. The mIoU fluctuated at 10%.

This is my code for training.

from ml3d.torch.models import kpconv

from ml3d.torch.pipelines import semantic_segmentation

from ml3d.datasets.s3dis import *

import torch

import open3d.ml as _ml3d

import sys, os

if name == 'main': os.chdir(sys.path[0])

cfg_file = "./ml3d/configs/kpconv_s3dis.yml"
cfg = _ml3d.utils.Config.load_from_file(cfg_file)

dataset = S3DIS(**cfg.dataset)

model=kpconv.KPFCNN(**cfg.model)

pipeline = semantic_segmentation.SemanticSegmentation(model=model, dataset=dataset,**cfg.pipeline)

pipeline.run_train()

Can anyone get the reported result? I would appreciate it if you can give me some suggestions.

Cheers,

Eric.

[shayan-nikoo]

On which S3DIS dataset are you running the training? The original Kp-Conv uses not-aligned version Stanford3dDataset_v1.2.zip, see here: https://github.com/HuguesTHOMAS/KPConv-PyTorch/blob/master/doc/scene_segmentation_guide.md

When you say it has 10% fluctuation, then what is your final mIoU? Are you comparing with the shared results from the original paper or from Open3d-ml? Can you share your results log for the last several epochs?

[Arno-Tu]

Hello, I saw that you mentioned in your question that you successfully reproduced the RandLA-Net network on S3DIS dataset. I would like to ask what are the network parameters you set? When I recently trained S3DIS data with RandLA-Net, I got all the loses value as NaN