Segmentation verification module for finalizing multi-label AI segmentations

[cpinter]

Draft Status

Ready - team will start page creating immediately

Category

Segmentation / Classification / Landmarking

Presenter Location

Online

Key Investigators

• Csaba Pintér (EBATINCA, Spain)
• Andrés Díaz Pinto (NVidia, UK)
• Dániel Palkovics (Dent.AI Medical Imaing, Hungary)
• ? Anyone interested ?

Project Description

Creating multi-label segmentation models is a challenge, because the deep learning model will try to segment all the structures that it was trained on, even if some are missing. This issue is present for vertebra segmentation, but most predominantly teeth segmentation. The Dent.AI 3D Guide dental implant planning software contains a tool for fixing the most typical errors in a user friendly way. It would be great if this tool was published as an open-source Slicer module.

Objective

1. Create an open-source Slicer extension from the Segmentation Verification widget from the Dent.AI 3D Guide custom app

Approach and Plan

1. Discuss the current Segmentation Verification widget. Is the current design satisfactory for a more generic use? Is there any other issue that needs to be handled?
2. Take the existing Segmentation Verification widget from the custom app and create a Slicer module from it
3. Create an extension containing this module

Progress and Next Steps

1. Describe specific steps you have actually done.

Illustrations

Background and References

• DeepEdit paper by Andrés
• Dent.AI 3D Guide software

[fedorov]

We have ~125K chest CTs segmented with TotalSegmentator, with everything publicly available. Of course, those segmentations have not been manually verified. Would this be an interesting use case to consider?

[cpinter]

I think any use case is interesting that fits the general pattern. The idea behind the current implementation of this widget is that you go through the segments one by one (Next button -> segment turns red), and check if it has some of the following issues:

• Label is simply wrong
• A target label is split into two segments (or more)
• The segment contains part of an adjacent segment. E.g. in the screenshot, the side of the red tooth would actually belong to the one next to it, so if the user clicks "Cut and merge part with..." then you can cut the part that should not be red using scissors, and it will be made part of the adjacent segment

I think the last case is somewhat special to teeth, because a full body segmentation will not allow you to fix such thing with scissors (other segments behind etc). Also there mught be other "typical" issues that can be fixed by a new action. But one point of this project is to discuss a potential variation of this tool that is more generically usable, so let's discuss!

[fedorov]

Agreed! It would be interesting to at least try this tool out with TotalSegmentator results we have.

cc: @vkt1414 @deepakri201

[github-actions[bot]]

Project Page Pull Request Creation

:white_check_mark: COMPLETED: See https://github.com/NA-MIC/ProjectWeek/pull/1094