Add tutorials

[dpascualhe]

Add notebooks with simple tutorials. E.g., evaluating a UNet trained with pytorch against Rellis3D.

[Ravish990]

Hello @dpascualhe I am Ravish. I am studying at Polaris School Of Technology , where I am in first year . I want to contribute to the organization . I am very excited and ready to put my efforts in contributing to the organization . Can you please assign the issue to me to get started ?

[sergiopaniego]

Hi @Ravish990, thanks for your interest in contributing!

You can start by installing the package and working on the issue by suggesting your ideas on how to address it😄

[Ravish990]

Thank you, sir, for your guidance. I have started exploring this area and am excited to learn new things😃. I look forward to contributing to the organization with my full potential.

Thanks sir for you time and support .

[SakhinetiPraveena]

Hello @dpascualhe ,myself Praveena, currently working full time as a full stack application developer. I am starting out my open source journey. I want to contribute to this organisation for GSOC'25. Looking forward to learning and making significant contribution to the project.

I am trying to install and build the package following through Installation.md, I couldn't find the file CMakeList.txt hence cmake .. is giving me an error. I am using a mac machine. Could you help me navigate through the error?

[dpascualhe]

Hi @SakhinetiPraveena, thank you for your interest!

What docs are you using? Maybe you are referencing the docs for the previous DetectionMetrics version?

Our current codebase is purely python based and is distributed as a Python library + CLI, so there is no cmake involved. This info is available both in the README and our website. Relevant links:

• Installation instructions: https://jderobot.github.io/DetectionMetrics/v2/installation/
• Notebook tutorial: https://github.com/JdeRobot/DetectionMetrics/blob/master/examples/tutorial_image_segmentation.ipynb

[SakhinetiPraveena]

Thanks a lot for clarifying. I see that you want someone to review and give feedback on the tutorial. I'll get on it.

[SakhinetiPraveena]

Hi @dpascualhe,

I am using a mac machine and CUDA isn't supported on apple hardware. Can you suggest an alternative if it's feasible. If not I will shift to using a linux vm.

Also to give you an update I was able to successfully install the library. I was facing the issue with CUDA when I was trying out the example. Once I find an alternative for that I should be able to start contributing. If there is a requirement for any specific tutorial you want me to create, I can get started on it or I can start with adding tests since no tests were added so far.

[dpascualhe]

Hi @dpascualhe,

I am using a mac machine and CUDA isn't supported on apple hardware. Can you suggest an alternative if it's feasible. If not I will shift to using a linux vm.

Hi @SakhinetiPraveena , CUDA is recommended for acceleration, but it shouldn't be a hard requirement. Have you tried simply installing the latest pytorch version and running the notebook? It should work fine (slower tho). Otherwise, I believe pytorch supports GPU acceleration in MacOS although I don't know much about it (https://pytorch.org/docs/stable/notes/mps.html). Testing out and updating DetectionMetrics to support GPU acceleartion in MacOS would also be a great contribution 😄

If there is a requirement for any specific tutorial you want me to create, I can get started on it or I can start with adding tests since no tests were added so far.

As of now, we only have a tutorial for image segmentation with pytorch. Potential tutorials to be developed are LiDAR segmentation, dataset conversion/merging, image segmentation with tensorflow... The scripts in the examples directory can give you some inspiration 💡

[SakhinetiPraveena]

Hi @dpascualhe , I tested out DetectionMetrics using image segmentation tutorial in my mac machine. I have added GPU acceleration for MacOS using MPS and raised the Pull Request .

torch.jit.load() was also throwing an error while running on mac because of device mismatch. Adding the map_location argument explicitly resolves the issue and makes sure the model is loaded to the right device in all cases.

Do let me know if I need to make any modifications to this. Looking forward for your feedback!

[nithika987]

Hello @dpascualhe, this is Nithika Balaji, and I'm currently beginning my journey in open source and excited to contribute to the organization in GSoC 2025.

I am facing some difficulties in running the example tutorial,(https://github.com/JdeRobot/DetectionMetrics/blob/master/examples/tutorial_image_segmentation.ipynb)

My error: "No valid deep learning framework found" when I try to import TorchImageSegmentationModel, even though I have done all necessary installations

• OS: Windows 11
• Torch version: 2.2.2+cpu
• CUDA available: False
• Open3D version: 0.19.0

Debugging attempts:

1. I edited init.py in models to print the error and got the following:

• Torch not available No module named 'open3d._ml3d'

2. When I tried: python -c "import open3d.ml.torch as ml3d"

• error: Exception: Open3D was not built with PyTorch support!

Could you guide me on how to resolve this? Thanks in advance!

[dpascualhe]

Hello @nithika987 ! Thanks for testing out our tutorial 😄

Regarding Open3D was not built with PyTorch support!, it seems that Open3D-ML is not supported in Windows as of now: https://github.com/isl-org/Open3D/issues/6519.

However, if you are not working LiDAR data (which you aren't as the tutorials only covers image segmentation), you shouldn't have any issues. We have recently merged some changes to wrap up every Open3D-ML import clause and avoid crashing if it is not available: #278

Could you debug and share the exact line that is yielding the exception reported?

Thanks!

[nithika987]

Hi @dpascualhe thanks for getting back! The steps I followed for the setup: mkdir .venv python -m venv .venv python.exe -m pip install --upgrade pip pip install torch==2.2.2 torchvision==0.17.2 pip install -e . (Additionally I had to connect to Jupyter through: pip install ipykernel python -m ipykernel install --user --name=.venv --display-name "Python (.venv)"
) Versions installed: Torch: 2.2.2+cpu | TorchVision: 0.17.2+cpu | Open3D: 0.19.0

The error: Torch not available No module named 'open3d.ml3d' Tensorflow not available No module named 'tensorflow' Exception: File ~\jderobot\DetectionMetrics\detectionmetrics\models_init.py:22 19 print("Tensorflow not available",e) 21 if not REGISTRY: ---> 22 raise Exception("No valid deep learning framework found")

Exception: No valid deep learning framework found

Debugging:

1. I got the below even though pytorch installed before open3d, open3d not automatically detecting pytorch installation python -c "import open3d; print(open3d._build_config)"

{'BUILD_TENSORFLOW_OPS': False, 'BUILD_PYTORCH_OPS': False, 'BUILD_CUDA_MODULE': False, 'BUILD_SYCL_MODULE': False, 'BUILD_AZURE_KINECT': True, 'BUILD_LIBREALSENSE': True, 'BUILD_SHARED_LIBS': False, 'BUILD_GUI': True, 'ENABLE_HEADLESS_RENDERING': False, 'BUILD_JUPYTER_EXTENSION': True, 'BUNDLE_OPEN3D_ML': False, 'GLIBCXX_USE_CXX11_ABI': True, 'CMAKE_BUILD_TYPE': 'Release', 'CUDA_VERSION': '', 'CUDA_GENCODES': '', 'Tensorflow_VERSION': '', 'Pytorch_VERSION': '', 'WITH_OPENMP': True}

2. I tried installing requirements using " https://github.com/isl-org/Open3D-ML/blob/main/requirements-torch.txt " from official repo, but still facing same error

Is there a specific way to install open3d ML to recognize pytorch? Or should I just switch to linux VM? Would appreciate any suggestions! Thanks!

[nithika987]

Hi @dpascualhe Nithika Balaji here again, I've identified the issue causing the error: "Torch not available No module named 'open3d.ml3d'"

Issue 1:

• torch.py has => from detectionmetrics.models import torch_model_utils as tmu ; import detectionmetrics.utils.lidar as ul
• Both import open3d._ml3d : which is not supported in Windows

Solution: Creating a mock module: mocks the open3d._ml3d module on Windows by injecting a custom MockModule into sys.modules and modifying sys.meta_path to ensure Python finds it. This allows code that imports open3d._ml3d to run without errors, even if the actual module is missing.

Eg: open3d_ml3d_patch.py ->displays an error only when ml3d functions called in Windows (works as usual in Linux), doesn't raise error on just importing open3d._ml3d in Windows. Must be imported into the init.py files.

Test 1: python -c "from detectionmetrics.models import TorchImageSegmentationModel; import torch; import open3d._ml3d; print(torch.version)"

Output: 2.2.2+cpu

Issue 2: tutorial_image_segmentation.ipynb in examples=> When system doesn't support CUDA

• Device compatibility in PyTorch, while calling TorchImageSegmentationModel(), model was saved with CUDA (GPU) and I'm running it on CPU. The error occurs because aten::empty_strided is trying to use CUDA, but my system does not support it.

• Solution: Converting the model to CPU before loading it.

[nithika987]

Hi @dpascualhe,

Regarding my fix for Issue 1 (mocking open3d._ml3d on Windows), does this approach seem feasible to you? If so, would it be appropriate for me to raise a PR for it?

Looking forward to your thoughts!

[dpascualhe]

Hi @nithika987 ! Could you please make sure to pull the latest commit from the master branch? I think you might be facing some issues that should be solved by now.

Issue 1:

• torch.py has => from detectionmetrics.models import torch_model_utils as tmu ; import detectionmetrics.utils.lidar as ul
• Both import open3d._ml3d : which is not supported in Windows

In the latest version, all imports for Open3D-ML are wrapped in try/except clauses like these:

try:
    from open3d._ml3d.torch.models.kpconv import batch_grid_subsampling, batch_neighbors
except Exception:
    print("Open3D-ML3D not available")

You should see a trace saying 'Open3D-ML not available', but the exception should not be raised. Links to all relevant lines: https://github.com/JdeRobot/DetectionMetrics/blob/ef8942f3a229d3a553c7d14dfb2009537fbcf148/detectionmetrics/models/torch_model_utils/o3d_randlanet.py#L4-L7 https://github.com/JdeRobot/DetectionMetrics/blob/ef8942f3a229d3a553c7d14dfb2009537fbcf148/detectionmetrics/models/torch_model_utils/o3d_randlanet.py#L4-L7 https://github.com/JdeRobot/DetectionMetrics/blob/ef8942f3a229d3a553c7d14dfb2009537fbcf148/detectionmetrics/models/torch_model_utils/init.py#L4-L7

Issue 2: tutorial_image_segmentation.ipynb in examples=> When system doesn't support CUDA

• Device compatibility in PyTorch, while calling TorchImageSegmentationModel(), model was saved with CUDA (GPU) and I'm running it on CPU. The error occurs because aten::empty_strided is trying to use CUDA, but my system does not support it.

#279, which is already merged, should solve said issue, as we are now explicitly setting the map_location parameter to whatever device is available (preference order CUDA, MPS, cpu)

Please ensure that you are using the latest code and let me know if your issues are solved!

[nithika987]

Thank you so much for the clarification @dpascualhe! And sorry for the trouble :(

[dpascualhe]

Thank you so much for the clarification @dpascualhe! And sorry for the trouble :(

no problem at all! Let us know if you manage to make it work 😄

[nithika987]

Hii @dpascualhe I'm not facing any issues now, thankss !!

[jayzalani]

Hey! I did check out the Databases that are supported but did not see Rellis3d can i try to add this along with its tutorial? @dpascualhe

[dpascualhe]

Hey! I did check out the Databases that are supported but did not see Rellis3d can i try to add this along with its tutorial? @dpascualhe

Hi @jayzalani ! RELLIS-3D is supported (Compatibility). In fact, the current tutorial makes use of RELLIS-3D as the evaluation dataset. Feel free to give it a try and let us know how it went! 😄

[jayzalani]

ohh! thanks I was exploring this. That's great will explore this and try to build some good tutorial in This dataset. 😎

after some research..... so I saw some of the examples, and got idea for contributing a fine-tuning tutorial for semantic segmentation models with DetectionMetrics. Would a tutorial demonstrating how to fine-tune pre-trained models on challenging Rellis3D samples be valuable for the project? @dpascualhe

[dpascualhe]

Hi @jayzalani , sorry for the confusion, but I think you are looking into the docs for DetectionMetrics v1, which is no longer mantained. Your entrypoint should either be our updated website or the README. As stated there:

Now, we're excited to introduce DetectionMetrics v2! While retaining the flexibility of our previous release, DetectionMetrics has been redesigned with an expanded focus on image and LiDAR segmentation. As we move forward, v2 will be the actively maintained version, featuring continued updates and enhancements to keep pace with evolving AI and computer vision technologies.

Relevant links for the current version:

💻 Code 🔧 Installation 🧩 Compatibility 📖 Docs

Would a tutorial demonstrating how to fine-tune pre-trained models on challenging Rellis3D samples be valuable for the project?

Fine-tuning or other training stages are not the main focus of DetectionMetrics. Our goal is to unify the evaluation stage for different frameworks and datasets. In that sense, new tutorials could focus in either using other datasets or frameworks (as of now our tutorial uses PyTorch and RELLIS-3D), or exploring other tools in the repo like computational cost estimation.