TotalSegmentator
TotalSegmentator copied to clipboard
wasserth
Tell us what you are using TotalSegmentator for
It would be really helpful for us to now who is using TotalSegmentator and what they are using it for. Therefore, we invite you to comment on this issue: What are you using TotalSegmentator for? What classes are you using? Are there any classes you are missing (we can try to add them in future releases)? Who are you (only if you want to disclose this)?
@wasserth HI sorry to disappoint, I'm am just an enthusiast amateur in medecine, I have no serious use for this software currently, still it is a potentially life saving advance in medecine. Noob question: I want to use the online version and give it as input a CT scan of a leg. So my question is: what are the supported formats for the CT scan single file? I know it must be a zip but once uncompressed, can it be a regular image (JPEG, PNG) or does the image needs to conform to a specific format? E.g can I input this? https://www.researchgate.net/figure/Patient-Abdominal-CT-scan-Normal-diameter-appendix-with-no-evidence-of-inflammation-of_fig2_323371468/amp
Hi, the input needs to be a DICOM image (compressed inside of a zip) or a NIFTI image. JPEG or PNG does not work. You can find a example CT file here: https://github.com/wasserth/TotalSegmentator/tree/master/tests/reference_files
Hi Jakob @wasserth ,
Thank you for letting us try this great program. We will be using this for lung, lobe, and rib cage segmentation and add these as additional components for our surgical planning.
Our project is to further improve our lung segmentation and analysis platform within 3D Slicer.
Are there any plans to include segments, more detailed airways, or lung vessels in totalsegmentator?
Best Rudolf
Great to hear that our tool is working well for you. We are currently working on adding airways and vessel segmentation for the lung as well as adding pneumothorax segmentation. What are you meaning by "segments"?
That is great news. If you want us to test any airways or vessels please let me know. We have a well-working airway segmentation tool at hand within 3D Slicer.
In addition, we plan to set up an ML active learning platform using IDC data soon. This will be based on DICOMWeb, 3D Slicer, MONIALabel, and lungmask. If we could also work with TotalSegmentator in this context it would be awesome.
The lung segments (10 on the right, 8 on the left side) would be the next and final substructures of lobes - they are not clearly divided by structures like fissures but usually pyramids of tissues originating from the branches of segment bronchi and pulmonary arteries (not veins). The segments are clinically important because in lung surgery we have more and more switched to segmentectomies in recent years for small lung cancers. For that one needs to know whether or not the tumor is located centrally in one or more of these segments. Segment borders are always approximations on CT and can later be intraoperatively defined by lack of perfusion (if you cut the vessels) or lack of segmental ventilation (if you cut the bronchi).
(wikipedia)
Thanks for the explanation. We have no plans so far to add those segments. It seems that it would be quite a bit of effort to manually draw those segments since the borders are not clearly visible in the image (but are only defined by the vessels).
Understand. How many labeled datasets would you need for a sufficient segment training process with TotalSegmentator?
In the beginning I would roughly need 20 labeled datasets. Then I would train a model and predict the 1200 subjects from the training dataset, then those predictions would need to be corrected where necessary. Could you provide 20 labeled datasets?
I'll discuss it with my team, but we could probably include this plan in our current 3D Slicer lung segmentation project and provide those complicated labels of the 20 datasets within 6-8 weeks.
On which data would you like to work on here for the initial training? Could we choose the data or would you define them?
Sound good. Ideally you would pick 20 subjects from the TotalSegmentator dataset: https://zenodo.org/record/6802614
I does not really matter which subjects as long as they are from the training set (which subjects belong to which set can be seen in the meta.csv file which comes with the dataset).
I'll contact you about this on Thursday. Which communication do you prefer? Email or are you on Discord?
What a phenomenal tool, thank you for making it publicly available. I am very interested in some of the additional models you have added after publication of the original paper, especially the vessel segmentation models since there's very little public data out there for this problem!
Will you be posting metrics for these additional models, as you have for the original 104 classes? Was the same training set used? Will you eventually be releasing the additional annotations?
In the future we will also do evaluation for those additional models and release the labels. However, for now this is not very polished yet. For this reason I did not add them in the beginning, but after realizing that those models might already be quite helpful for other people I added them.
Hi, I found the solutioin very interessoing. I am evaluating the code. We want to use it in a project for planning cardiac interventions.
Hi, thank you for this amazing work, I was wondering if you thought about segmenting fat tissue (and differentiating perivisceral fat tissue from sub cutaneous fat tissue). we can consider making some annotation if you have interests in this project. Salim
I am currently working on adding sub cutaneous fat and visceral fat segmentations. But it will take a few more weeks until it is done.
That's cool, thank you for all the job done, really interesting project in which we hope to contribute in the future.
I'm currently working on a pre-clinical project at the University Hospital in Copenhagen, where we are interested in total tumor segmentation. We published an article about using the nnUnet for this task, with great results in improving time needed for physicians to do segmentations. Link to article: https://ejnmmires.springeropen.com/articles/10.1186/s13550-022-00901-2
Now we would like to combine the information of tumor segmentation with their location in the body, and I'll try to implement your organ segmentation model, to gain the knowledge of which organs the tumors are located in. This will allow physicians to quickly create an overview of the total tumor volume, as well as the location and volume of individual lesions.
Thanks for the great work!! It's highly appreciated. Kristian
I am going to test its results for segmenting ~150 whole-body CT scans of pediatric patients. My current interests are segmenting the lungs and possibly the bronchi and trachea. Any plans to train and validate this tool for segmenting CT scans of pediatric patients?
Lung lobe segmentation has recently become a reliable 2-minute process within 3D Slicer with Lung CT Segmenter calls of the TotalSegmentator AI extension.
Consequently, the free and open-source 3D Slicer Lung CT Analyzer extension (V 2.62) now includes lung lobe analysis of pulmonary infiltrates, lung collapse, and emphysema. A special emphysema result table and display are available which will be helpful in lung volume reduction surgery (LVRS) and the surveillance of COPD.
In this COPD case, 35% of the right upper lobe is affected, so this anatomical structure could be a primary target for LVRS.
The pathology is well detected with conventional thresholding in the TotalSegmentator rendered right upper lobe.
https://user-images.githubusercontent.com/18140094/208263189-a09c2c60-b747-4152-a975-6bd086969d12.mp4
Very excited to try out this tool, but I haven't yet had the chance to install the code.
Can you please comment on the output format of TotalSegmentator? NIFTII? Volumetric or 2D?
Thank you!
This is truly an impressive project, I can't even begin to fathom the amount of time and effort your team must have put into this, AND its all open source! 🙏🎉🎉🎉
On top of my mind, I already know a few PIs that have existing research/cro projects that needed this YESTERDAY. Given that contouring of organs are done manually in many projects (typically oncology, with different organ focuses, liver, kidney...), any contour from CAD would be a productivity booster/tremendous time saver for the image annotation team. Obviously, for clinical work, this will drop the turn around time for report generation/surgical planning, thus bringing quantitative image analysis one huge step towards clinical practice.
Below, I am sharing the dice computed from TotalSegmentator outputs versus manual contours in AMOS2022 (n=357) and Pediatric-CT-SEG datasets (n=~359, age ranging from 0 to 16) (** disclaimer, there could still be bugs in my assessment scripts, further no other datasets have 104 labels, thus i had to merge the 104 labels to the dataset's label set ** )
AMOS2022:
Pediatric-CT-SEG (with and without --fast flag):
Dice of liver in Pediatric-CT-SEG grouped by age (cc @owensca, since you mentioned you are interested using this for pediatric patients):
- scripts to generate the above available here.
- you can view other organs plotted by age here.
Happy New year, congratulations on your amazing work. I am working as a doctor in the craniofacial area; it is already very useful to me that totalsegmentator give automatically the brain volume. I am more interested in the face, which appears as a global area. Is it a chance that it could be segmented in the future? I am not a geek, and I hardly understand the technical names, but I am a good clinician, and I wonder how I could help with the development of this part of the process. The skeletal part is easily done, but the segmentation of other parts: eyeball, masticatory muscles, pharyngeal muscles, would be useful in the clinical research on CFM malformations etc...
I would like to use this software to segment the spine and vertebra out of dicom images, but sadly the software is not working correctly (either with docker or normal build).
I would appreciate any help!
Please open a new issue with your specific error message so we can try to fix your problem.
Happy New year, congratulations on your amazing work. I am working as a doctor in the craniofacial area; it is already very useful to me that totalsegmentator give automatically the brain volume. I am more interested in the face, which appears as a global area. Is it a chance that it could be segmented in the future? I am not a geek, and I hardly understand the technical names, but I am a good clinician, and I wonder how I could help with the development of this part of the process. The skeletal part is easily done, but the segmentation of other parts: eyeball, masticatory muscles, pharyngeal muscles, would be useful in the clinical research on CFM malformations etc...
Thanks for pointing to those structures. We will see if we can add the eyeball, masticatory muscles and pharyngeal muscles in a future release.