Using additional Features like Normals or Intensity

[Nerolf05]

Hello, I would like to classify 3d-data, using additional features, like normal (Nx3) or intensity (Nx1) to improve my results.

What do i have to change, except the number of input_channels for the model and the Data Loader? I have already updated the Data-set_Loader, which getitem method returns the point, label, normal and intensity, but now i don't know how to change the loss calculation. Especially, how to unwrap the batch into input and labels in the "model_fn.

Greetings

[erikwijmans]

You shouldn't have to change the loss function or the model_fn. Everything assumes that the point cloud is of shape [N, 3 + C] where each point is (x, y, z, ....). So if you do something like

xyz = Nx3 array of xyz locations
normals = Nx3 array
intensity = Nx1 array
label = the_label

return np.concatenate([xyz, normals, intensity], -1), label

in the dataloader, everything else should just work.

[Nerolf05]

Thanks a lot @erikwijmans for your fast answer

When i want to use composed transformations, like PointcloudRotate or PointcloudRotatePerturbation, i have to make sure, that only the metric data (normals) are transformed, as well, right?

Is there a possibility to save the final results for visualization?

Greetings Florian

[erikwijmans]

Yes, you'll need to make sure that rotate and perturb don't change RGB and intensity values, but do change normals.

There isn't any code to visualize the results in this repo, but it isn't too hard to do with Open3D: http://www.open3d.org

[Nerolf05]

Once more Thank you

I have few more questions:

I want to classify real world airborne Lidar-Data:

Data-Augmentation:

• Is there a special reason why you rotate, PointcloudPointcloudRotate, the single point-cloud around the y-Axis per default?

• Should the factor for the RandomInputDropout (87.5%) be reduced for real world data

• What is the usage of reducing the number of used points per cloud, DataLoader.randomize(). Right now, i' m struggling with a lack of trainings-/test-data. Would it be advisable to set the number to 100% ?

Results:

• Might the final weights be saved to apply the trained-net on unlabeled Data?
• Can i find the type 1 & 2 error somewhere?

Thanks in advance

Greetings

[erikwijmans]

Is there a special reason why you rotate, PointcloudPointcloudRotate, the single point-cloud around the y-Axis per default?

The negative y-axis is the gravity direction for that dataset IIRC. Allowing the network to learn about that bias seems fine/useful.

Should the factor for the RandomInputDropout (87.5%) be reduced for real world data

I have no idea! Sounds like a good question to test :)

What is the usage of reducing the number of used points per cloud, DataLoader.randomize()

It's just another method of data-augmentation. I have no idea if you should use 100% on real-world data, sounds like a good question to test :)

Might the final weights be saved to apply the trained-net on unlabeled Data?

There is nothing stopping you from doing that, but I don't know if it will improve results.

Can i find the type 1 & 2 error somewhere?

That isn't implemented anywhere.

[Nerolf05]

Hi @erikwijmans

Right now i am still trying to improve my results by using more features. For example machine learning features, like statistical sizes (co/variance) or some image values, which are equal for all points of one object. While evaluation all objects are assigned to class "0", whereby the model(inputs) returns for the prediction only "nan-values" after the first validation.

At the beginning i thought using the object-features for all points of the object might be the reason, but when i used one intensity for all points of one object (without additional features), i got better results. I also tried to scale the value-range of the features and changed the learning rate few times.

Do you have an idea what might be the problem?

Thanks in advance

Greetings :)

[erikwijmans]

NaNs suggest gradient spikes. Try a lower LR.

If you have a lot of points with the same feature, batch norm may do something bad. You can try GroupNorm instead if the LR isn't the issue.

[Nerolf05]

I have tried several LR but nothing changed.

Where do i have to implement the PyTorch GroupNorm? Will it replace the BatchNormMomentum-scheduler?

[erikwijmans]

The layers are defined here: https://github.com/erikwijmans/etw_pytorch_utils/blob/master/etw_pytorch_utils/pytorch_utils.py

GroupNorm has no momentum, so you won't need the momentum scheduler.

[Nerolf05]

Thank you very much for all your advises.

At the end, i want to ask about the rating of the results. Currently i am using a data-set with four classes (train: 4 x 116 = 464, test: 4 x 51 = 204) and get results like the following picture: Am i allowed to say my best epoch is my final result (91.18%) or do i have to calculate some statistical sizes like the mean or std-Dev. of the last x-epochs ? Is it correct to say the network is able to achieve 91.18% accuracy and after that epoch the network starts overfitting?

Thanks in advance