OHIF
[Feature Request] Display the label for the segment at the mouse location
What feature or change would you like to see made?
It would be helpful to be able to show the name of the segment at the pixel location pointed by the mouse in a tooltip, or in some "probe" area of the interface.
Why should we prioritize this feature?
Once the number of segments reaches couple of dozens (and it can easily be ~100 with the modern AI segmentation tools), it is quite difficult to know what is the label for a given segmentation in the viewer.
Especially for someone who does not know anatomy well - like me (and at least some AI segmentation tools developers, I dare say)! :-D
@dan-rukas @sedghi we are thinking we would need design developed/approved first here? Can you let us know? We would be interested to prioritize work on this, and I think this would be a very useful feature outside of IDC.
@fedorov I'm just waiting for a green light to start working on this one.
I deployed using this recipe we have been using in IDC https://github.com/ImagingDataCommons/idc-viewers-sandbox-gha-testing/blob/main/.github/workflows/deploy-ohif-v3-upstream-with-add-ons.yml, and I do not see this feature. Do I need to anything special to be able to use it?
@fedorov you have to activate it on the segmentation panel here:
(this is a screenshot from the version deployed by you)
Ok, I found it in my deploy. So now I have the following questions:
- Why is it in the Segmentation mode? This is a feature most valuable for the basic viewer, when we need static review of existing segmentations.
- I still cannot make it to work, even though it is enabled, see demo here: https://app.screencast.com/QqWys9ShkGRxK
- Is this setting exposed via the config file, so it can be enabled by default on deploy?
- Why is it in the Segmentation mode? This is a feature most valuable for the basic viewer, when we need static review of existing segmentations.
I've also added it to the basic viewer (longitudinal mode). It should be available on the same menu. Please let me know if it is not.
- I still cannot make it to work, even though it is enabled, see demo here: https://app.screencast.com/QqWys9ShkGRxK
That is weird. Did you get any errors in the console?
- Is this setting exposed via the config file, so it can be enabled by default on deploy?
No. I'll add that for you.
@fedorov can you share the url of the study you were looking at in the video?
@pedrokohler I do not see it in the basic viewer - but it might be that it is just well-hidden! :-D
Here's the URL (not sure if you would be able to open it, but you can fetch the study in question from IDC with pip install idc-index && idc download 1.2.840.113654.2.55.175589378701871887335124424440855375150): https://viewers-sandbox-gha-testing.web.app/viewer?StudyInstanceUIDs=1.2.840.113654.2.55.175589378701871887335124424440855375150
Indeed, there is an error in the console ("Segmentation mode" only)!
@fedorov I see. It's because you didn't click the load button up there. I'll see that it also works in this condition.
I'll also take a look at why it's not showing in the basic viewer.
Oh right the LOAD button! Am I the only one who is confused by it? Anyway, I think we are auto-loading it in IDC, so hopefully our users won't need to worry about it.
The use of the color assigned to the segment for the label annotation is witty, but it may not be very helpful, as some colors can blend with the image. Did you consider using bright yellow, or something like that, to make that label consistently prominent?
It was initially yellow, but Alireza requested it to be changed to the color of the segmentation.
I'll also add a new PR to cornerstone allow this to be configurable. This way everyone is happy.
but Alireza requested it to be changed to the color of the segmentation
@sedghi what are your thoughts about the result of that below?
I have seen that in other viewers to be the same color of the segment, maybe we need to drop a shadow behind it. or if it already have a shadow we increase it
@fedorov feel free to comment here https://github.com/OHIF/Viewers/issues/5190
It took me some time to find this - this feature was added as a new tool.
While testing this, I discovered that 1) the tool can be enabled even if there is no SEG in the view (I don't know if this is expected behavior); 2) if it is enabled without a SEG, it will trigger error popup and the following errors in the console.
yeah weird, it should check if there is a SEG, it is easy, we can add a new evaluator to check if SEG exists
Not really.... It is a complicated mess, unfortunately, due to the variety of factors:
- IDC production (as well as all other tiers) OHIF cannot display RTSTRUCT, due to https://github.com/ImagingDataCommons/ViewersV3/issues/95, so I cannot test it with that
- the above should be fixed now, but @igoroctaviano has not yet upgraded our fork
- he has not upgraded our fork, because we are still hopeful we can get fixes for #5082 in - I have zero idea where that is at the moment, and who is working on it, if any
- we do have a sandbox instance we can test easily against IDC content here https://viewers-sandbox-gha-testing.web.app/, and I have just updated it with the latest master, but for some reason I do not see that tool there at all...
Not quite sure if I can do anything about this, I can't debug this, so I plan to wait until IDC fork is updated by Igor, and will then test in IDC dev tier.
@sedghi now that we updated to 3.11.1, I tested it, and it is not working reliably. Sometime it shows the label (e.g., for this StudyInstanceUID 1.2.840.113654.2.55.10251457513072085864761297205518907029 in IDC), on some other occasions it does not show anything and I see the errors below (for this study 1.3.6.1.4.1.32722.99.99.173744694849724904145228624619597754035).
Once we update the viewer in prod tier, I will add links.
Works here: https://viewer.imaging.datacommons.cancer.gov/v3/viewer/?StudyInstanceUIDs=1.2.840.113654.2.55.192012426995727721871016249335309434385&SeriesInstanceUIDs=1.2.840.113654.2.55.305538394446738410906709753576946604022,1.2.276.0.7230010.3.1.3.313263360.15787.1706310178.804490
Doesn't work here: https://viewer.imaging.datacommons.cancer.gov/v3/viewer/?StudyInstanceUIDs=1.3.6.1.4.1.32722.99.99.173744694849724904145228624619597754035 (segmentation series "Segmentation"). I think the issue here might be that it does not contain recommended color in DICOM. But it should not break the code - the color is used in OHIF.
In addition, after testing this feature, I would like to mention it again that I do not think the current choice of using the same color for the text annotation as the color of the segment is useful. I do not see how using segment color for the annotation helps in any scenarios, but it most definitely makes the text difficult to read under some conditions. I would strongly prefer using the same prominent bright color for this annotation, same color no matter what is the color of the segment.
Also, I think it is a bug that the annotation does not disappear if I scroll and the cursor is no longer at the same label as it was at the start of the scroll.