CTK Visual DICOM roadmap

[ ] 1. Implement sending in C++ in CTK (i.e., adding ctkDICOMSendJob, ctkDICOMSendWorker, and ctkDICOMSend with underlying DIMSE DcmStorageSCU). This would allow the use of the background/parallel operations infrastructure for SEND as well.
[ ] 2. Add DICOMweb support.
[ ] 3. Handle the jobs queue in the scheduler by file (so we can restart the jobs/workers at application restart).
[ ] 4. Add data streaming from visual browser series widgets to Slicer volume nodes.
[ ] 5. Add support for DICOM frame set (GitHub issue, discussion).
[ ] 6. Test loading of series with an extra “localizer” slice (discussion link 1, discussion link 2). Test also Fluoro acquisitions.
[ ] 7. Restore the old advanced/examine UI for loading a series in the visual DICOM browser.
[ ] 8. Improve the visual DICOM browser (and CTK in general) to use styleSheet instead of QPalette for dynamic color changes. Ideally, the solution should:
1. Use one unique style file containing the stylesheet for all CTK classes.
2. Change the color of Qt widgets in CTK dynamically with the Property Selector (e.g., Dynamic Properties and Stylesheets).
3. Allow custom Slicer apps to change the style by simply rewriting and reloading the style file.
[ ] 9. Add a writing lock static variable in the ctkDICOMDatabase class.
[ ] 10. Implement smart management of inserts and memory when retrieving a series (e.g., add a customizable framesBatchLimit variable).
[ ] 11. Add infrastructure to set the maximum number of concurrent workers per server (currently, it is per job type).
[ ] 12. Update retry time delay to be exponential with a factor of 2/3 (with some randomness). The current default of 100ms is insufficient for server recovery. Replace the maximum number of retries (default: 3) with a maximum waiting time (defined in the GUI server settings). Add a timeout warning in the UI and ensure the string is clear.
[ ] 13. Allow disabling automatic prefetch of the full series (current behavior). If the option is active, retrieval of all frames should occur only when selected series are manually loaded.
[x] 14. Replace the Patients list as tabs in the tab control of the Patient widgets with the old widget from the ctkDICOMBrowser (first one, only the patient list). Add an option to switch between them.
[x] 15. Improve UI clarity regarding querying PACS servers vs. filtering the local database.
[x] 16. Address UI performance issues when importing a large number of patients (>500, >5000 series).
[x] 17. Add a button to enable full-screen mode in the visual DICOM browser.
[x] 18. Apply thumbnail size settings changes without requiring a restart.
[ ] 19. Add a feature for force download of series/study:
- Add a checkable setting (default: False). If any study is 3 hours old (based on DICOM study date), force redownload every time a query is performed (not for local search).
- Enable monitoring (right-click action) for a configurable time. Automatically fetch and update the UI if any series or instances in a series are uploaded to PACS. Add UI icons to indicate updates. For series, check if new instances have been added to the PACS.
[ ] 20. Add a feature to check a folder containing DICOM files when clicking the query/retrieve lens button. If files are present, import them and clean the folder. Create a lock file in the DICOM folder with the PID of the session accessing/deleting the files. Investigate using a watcher for automatic import. Treat the local folder as a "local connection" (like a server) to allow adding multiple folders in settings. Ensure the infrastructure is protocol-independent.
[ ] 21. Fix rendering for Segmentation DICOM objects in the ctkThumbnail class.
[ ] 22. Add an option in settings to order series in the study widget by modality.
[ ] 23. Automatically load the source volume when loading a segmentation.
[ ] 24. Show a popup warning when loading a series that is unsupported (or supported with an extension).
[ ] 25. Add support for DICOM TLS security.
[ ] 26. if the thumbnail does not render (e.g. SEG), do not create a thumbnail square. Also place the not rendered ones in new rows after the rendered ones (CT/MRI...)
[ ] 27. the thumbnail renderer should be made pluggable (similarly to the DisplayedFieldGenerator rules). Each registered thumbnail renderer could get a chance of inspecting a frameset, provide a confidence value, and the renderer with the highest confidence value would be used to generate the thumbnail.
[ ] 28. metadata dialog enhs: maybe better to have it directly in the main layout. would be great to be able to toggle both the thumbnails rendering and tags tree in the metadata viewer. It would be nice to have more clear metadata regarding patient, study and series in the dialog copying metadata tag/values etc... maybe with just one click (hovering -> icon for copying) or a dropdown menu double click -> can be the action on the dropdown menu and in default we can have the copy value (other: navigate to website, copy all, ....)

References:

old roadmap: https://github.com/commontk/CTK/issues/1162

Apr 23 '25 10:04 Punzo

Replace the Patients list as tabs in the tab control of the Patient widgets with the old widget from the ctkDICOMBrowser (first one, only the patient list). Add an option to switch between them.

This would be a much welcomed change - @rkikinis and I were completely confused by the current interface.

How about also adding an option to offer the same organization as in the current DICOM Browser, showing also study and series panels?

Oct 09 '25 16:10 fedorov

Replace the Patients list as tabs in the tab control of the Patient widgets with the old widget from the ctkDICOMBrowser (first one, only the patient list). Add an option to switch between them.

This would be a much welcomed change - @rkikinis and I were completely confused by the current interface.

How about also adding an option to offer the same organization as in the current DICOM Browser, showing also study and series panels?

I am not sure to have understood, I am working now on points 14-16 these weeks, so feedback is very welcome to shape the refactoring of the UI.

Would you like to have all the studies and series in 2 separate panels like the old browser? that would clash with the current design, in which those two panels are essentially merged into one, and each item is a study and below that item there are all the thumbnails for the series for that study.

Oct 09 '25 17:10 Punzo

Would you like to have all the studies and series in 2 separate panels like the old browser? that would clash with the current design, in which those two panels are essentially merged into one, and each item is a study and below that item there are all the thumbnails for the series for that study.

Yes, I was asking if this would still be possible. I thought that this new browser will eventually deprecate the current one. If this is the case, it would make sense to deliver the traditional interface Slicer users have been using for 10+ years. But if both modules will co-exist, then I guess it is not a priority.

Oct 09 '25 17:10 fedorov

@Punzo could you post a few most recent screenshots?

@fedorov could you describe how you ideally use the DICOM browser? Currently, the visual DICOM browser is mainly for patient-oriented workflows (where you need to find a specific patient and then access all his/her data), but it should not be hard to make it better support data-oriented workflows (where you work with data of many patients at once). Maybe it would be enough to just allow multiple selection in the patient browser.

Oct 09 '25 18:10 lassoan

could you describe how you ideally use the DICOM browser?

@lassoan @Punzo this is a complex question - I am not sure I can cover in detail how I use it in different situations, it is just part of life at this point! I am very much used to the current DICOM Browser and learned to live with it. There are things I would like to improve, but for the most part, I learned to work around the limitations, and it works for me. I am a bit worried that most users may feel the same way, and may not realize there is a plan to replace it with DICOM Visual browser.

What do you think about organizing a virtual session where you would informally demonstrate what you have, discuss the roadmap, and hear from the users about possible concerns? I would be interested to attend, and knowing @rkikinis, I think he would join too. It would be more efficient at this stage I feel than back-and-forth here or on the forum.

Oct 09 '25 18:10 fedorov

Maybe we could do this at the next Slicer developer meeting (Tuesday 10am Eastern)? Would you be all available?

Oct 09 '25 18:10 lassoan

Yes, I can join, and so can Ron!

Oct 09 '25 19:10 fedorov

Yes, I was asking if this would still be possible. I thought that this new browser will eventually deprecate the current one. If this is the case, it would make sense to deliver the traditional interface Slicer users have been using for 10+ years. But if both modules will co-exist, then I guess it is not a priority.

@fedorov Ideally, the visual DICOM browser could eventually replace the old one, but I think that should only happen once it matches all the features of the traditional browser, in addition to the new capabilities. This transition will likely take time—similar to how the Plotting/Charting Widgets were gradually adopted years ago.

With the current design for the visual DICOM browser, separating the UI into two distinct panels for studies and series is challenging. As Andras suggested, supporting multi-patient selection and merging the resulting studies/series could help address some workflow needs, but we can discuss this further during the videocall.

From a technical perspective, the new design that I am working on for the refactoring is focused on interactive performance for very large databases: A) a patient-level model/view/delegate for all patients, B) a study-level model/view/delegate for each patient, C) a series-level model/view/delegate for each study. This separation was chosen for perfomances (you never have models with a very large quantities of data), but it can be used only with the visual dicom borwser idea to merge studies and series panels.

One way could be to have two modes:

the one that I am using (series/studies) merged (showed in the video below) with hierachical models infrastructure.
create 2 new proxy models to get all the studies and series data from the hierachical model infrastructure from (1). These will be connected to 2 new views/delegates for separated panels rendering. But it would be a lot of additional work and I am not sure I have the dev time for it. Again let's talk at the Slicer meeting.

@Punzo could you post a few most recent screenshots?

@lassoan here a very preliminary preview video of current status with a couple of studies from IDC (I refactored the series and studies levels, next week I will address the patient level).

https://github.com/user-attachments/assets/b7341441-a73b-4b1b-9a65-41b7acb10fb9

I tested also importing 5 GB of dicom and 1 patient with hundreds of studies/series. The perfomance were fully interactive!

branch is at https://github.com/Punzo/CTK/tree/visualDICOMBrowserUIRefactor

detailed to do:

currently series selection is per study, it should be per patient. Also study header items selection should be similar to the previous widget implementation (button to select/deselect all series in the study)
refactor the patient level UI with model/view/delegate pattern (allow multi-patients selection and find a better solution for the UI respect to the tab widget)
restore right click menus
fully integrate new UI with scheduler logic (query and retrieve)
test perfomances of query/retrieve for a large number of patients with a large number of studies/series
add automated testing for new UI classes

Maybe we could do this at the next Slicer developer meeting (Tuesday 10am Eastern)? Would you be all available?

good for me as well

Oct 10 '25 16:10 Punzo

Looks great @Punzo - talk to you at the Slicer weekly meeting!

Oct 10 '25 16:10 lassoan

the refactoring is focused on interactive performance for very large databases I tested also importing 5 GB of dicom and 1 patient with hundreds of studies/series.

When you say that, do you mean "very large" for a single patient, or "very large" for the entire dataset across all patients?

I would not consider 5GB to be large in either case. What do you consider "large"?

Oct 10 '25 21:10 fedorov

the refactoring is focused on interactive performance for very large databases I tested also importing 5 GB of dicom and 1 patient with hundreds of studies/series.

When you say that, do you mean "very large" for a single patient, or "very large" for the entire dataset across all patients?

I would not consider 5GB to be large in either case. What do you consider "large"?

What is the typical size of a DICOM database that you would open locally with 3D Slicer? And what will be an extreme case? More than 100 GB? And in terms of number of patients/studies/series?

In principle the new design I used it should scale and provide interactive performance. I still need to do quantitative benchmarks and performance profiling (I just finished this part of refactoring today) and I tested just briefly on some data that I had locally. On my side I was already happy because the previous implementation, widget-based, of the Visual DICOM browser had already performance issues for that size/number of studied/series (which we all agree it was very bad).

On Monday I will do more extensive testing, and then I will start refactoring the patient level. I will try to have something very preliminary for Tuesday that we can discuss.

Oct 10 '25 23:10 Punzo

@lassoan @fedorov I have finsihed the patient level refactoring:

tested with:

150 GB data from IDC -> 1011 patients, 1310 studies, 10k+ series.
1 patient (15 GB) with 71 studies and 354 series.

and perfomances looks good.

To Do list:

[x] (1) currently series selection is per study, it should be per patient. Also study header items selection should be similar to the previous widget implementation (button to select/deselect all series in the study)
[x] (1a) when selecting series, sometimes there some weird temporary clipping of the series UI
[x] (2) refactor the patient level UI with model/view/delegate pattern
[x] (3) tested with local database of 150 GB donwloaded from IDC

Oct 14 '25 13:10 Punzo

updated To Do (feedback from slicer meeting 14/10/2025):

[x] optimize the series table view (look also to OHIFv3 UI).
[x] easy way to change (ctrl +/-/scroll + some UI) thumbnail size (current small, medium, large are ok) and it should be dynamic.
[x] check for highdpi monitor thumbnail generation size.
[x] Allow to not render thumbnails, i.e. thumbanil size set to none.
[x] series selection hovering rendering is missing.
[x] For patient selection have both a tab and list selection. allow multi-patients selection in list mode
[x] patient filtering: only one patient, do not show the patient tab/list selection
[x] patient filtering: no patients found. Render some text telling the user nothing found.
[x] patient filtering: up to 5 patients -> use tab like UI, more use the list automatically. Have a button also to switch between them. (... button)
[x] patient filtering: for the patient list include added time metadata and sort the list with it
[x] in list mode/multi selection merge study/series models from all selected patients in a merged study model.
[x] restore context menus at right click. We can have also 3 dots icon showing on items when hovering them to open context menus.
[x] fully integrate new UI with scheduler logic (query and retrieve -> status report and progress bars). Remember the allowed server. Lense icon use only the cloud one to query and retrieve (local database is now showed automatically, no need to click the button anymore).
[x] test perfomances of query/retrieve for a large number of patients with a large number of studies/series
[x] add tests for new UI classes
[ ] add user documentation for the visual dicom browser (e.g. in https://github.com/Slicer/Slicer/pull/8786)
[x] more clear button in Slicer to activate the visual dicom browser
[x] arrow to close the module panel and show the visual dicom browser on a larger window.
[x] it would be nice if patient selection from the old dicom browser propagate into the visual one. If it is not possible at least provide a warning to the user. (for the user the fact that the 2 browsers are independet is highly counter-intuitive)

feedback from discussions below:

[x] add more modalities in the filtering options
[x] date filtering, needs start and end date calendar UI
[ ] if the thumbnail does not render (e.g. SEG), do not create a thumbnail. Also place the not rendered ones in new rows after the rendered ones (CT/MRI...) -> moved to main roadmap
[ ] make the thumbnail renderer should be made pluggable (similarly to the DisplayedFieldGenerator rules). Each registered thumbnail renderer could get a chance of inspecting a frameset, provide a confidence value, and the renderer with the highest confidence value would be used to generate the thumbnail. -> moved to main roadmap
[x] make filters section support both horizontal (current) and vertical (filters stacked one over the other) mode when resizing the window.
[x] in Slicer core change dicom module UI layout and make it that the user can resize/move the dicom viewers as dockable widgets or a simple switch between horizontal (we are using this now in the slice viewer) and vertical layout (module UI). So the user can either use horizontal layout to view a lot of data or in vertical layout (which saves space) to view both the viewer and Slicer for series visualization.

feedback from videocall with @lassoan @fedorov on 07/11/2025

[x] multi selection patient in list mode, should render patient ID, Name, Birth Date in the studies header.
[ ] metadata dialog enhs: -> moved to main roadmap

maybe better to have it directly in the main layout.
would be great to be able to toggle both the thumbnails rendering and tags tree in the metadata viewer.
It would be nice to have more clear metadata regarding patient, study and series in this viewer
copying metadata tag/values etc... maybe with just one click (hovering -> icon for copying) or a dropdown menu
double click -> can be the action on the dropdown menu and in default we can have the copy value (other: navigate to website, copy all, ....)

[x] series modality string over thumbnail -> use background behind the text instead of shadow with high opacity (~230).
[x] for modalities that do not render, remove the "document" icon, just render the modality text, e.g.: SEG
[ ] support for TLS -> moved to main roadmap

Oct 14 '25 15:10 Punzo

@Punzo I think we discussed that it is highly counter-intuitive that the database that populates visual browser is completely independent from the current browser. At the very least, there should be a prominent warning about this, which might make sense to pop whenever user toggles visual browser checkbox.

Oct 14 '25 15:10 fedorov

@Punzo I think we discussed that it is highly counter-intuitive that the database that populates visual browser is completely independent from the current browser. At the very least, there should be a prominent warning about this, which might make sense to pop whenever user toggles visual browser checkbox.

ok, I have updated point 8

Oct 14 '25 15:10 Punzo

feedback from the Slicer meeting 14/10/2025:

support for TLS

add documentation regarding patient-based/clinical dicom browser vs research needs.

show added last time metadata

more clear button in Slicer to activate the visual dicom browser

change the series tbale view to look like OHIF v3 (if the thumbnail does not render, do not create a thumbnail)

easy way to change thumbnail size (current small, medium, large are ok) and it should be dynamic

arrow to close the module panel and show the visual dicom browser on a larger window.

it would be nice if patient selection from the old dicom browser propagate into the visual one. If it is not possible at least provide a warning to the user. (for the user the fact that the 2 browsers are independet is highly counter-intuitive)

if I missed any point, please let me know @lassoan @fedorov @jcfr @sjh26 @markasselin @pieper . It would be nice to continue to talk the needs for more research based workflow.

Oct 14 '25 15:10 Punzo

@lassoan I have implemented most of the feedback (quality of the video is a little bit low to fit the 10MB github limit):

https://github.com/user-attachments/assets/1fef9675-5c27-4473-9d9b-6e249b7f7d43

tested on a subset of 150GB database from IDC. main still to do

implement context menu
integrate query and retrieve in the new UI
integrate in Slicer core

status details are above: https://github.com/commontk/CTK/issues/1230#issuecomment-3402330952

Oct 24 '25 20:10 Punzo

I think this looks much better!

I am still confused what is the motivation for showing the icon in place of the missing previews? Those icons take huge amount of space while providing zero value.

I personally think the design decision made in OHIF is much better to show just the description whenever preview is not available.

Oct 27 '25 17:10 fedorov

I think this looks much better!

thanks!

I am still confused what is the motivation for showing the icon in place of the missing previews? Those icons take huge amount of space while providing zero value.

I agree, I never particularly liked that workaround I used already from the previous version of the UI.

I personally think the design decision made in OHIF is much better to show just the description whenever preview is not available.

I really like the OHIF solution, but I had few issues why I did not already implemented it in this UI version last week (brainstrom them and coming up with solutions will be very nice):

OHIF panel for the study/series navigation is mostly a vertical layout, so it makes sense to put any item modality without a thumbnail as a new line object with only text. But in the visual dicom browser the layout would be for the most of time horizontal and we need a solution to fully cover the horizontal space.
I have already implemented for the thumbnail zoom a none value for the thumbnail dimensions and this how it looks like now (this is already implemented):

So I was thinking something like this (this is not implemented):

However I have a couple of technical issues for this implementation: A) I need to hardcode in CTK for which modality the rendering works. This because I need to know when creating the table where to place the items. Probably this is not a major issue for anynone, and probably it would be good in the ctkThumbnailGenerator to know for which modality works. B) since we have study/series filtering embedded in the model/view architecture, we have the data in a model and then we have a proxy filtering model. The ordering and the placing of UI element for the SR,SEG(,..) in new lines after the CT, MRI (,...) elements will be quite complex (essentially the indexing) and I have to think how to do it.

For these 2 reasons (mostly for B) I have not prioritized it (over the query/retrieve, etc...), but if you think is essential @lassoan @fedorov I will try to have a solution for (B).

Oct 28 '25 13:10 Punzo

@Punzo thank you for considering the suggestions! I definitely like the mockup you shared over the demo in the video!

OHIF panel for the study/series navigation is mostly a vertical layout, so it makes sense to put any item modality without a thumbnail as a new line object with only text. But in the visual dicom browser the layout would be for the most of time horizontal and we need a solution to fully cover the horizontal space.

Indeed, but I would like to re-iterate what I tried to suggest during the call...

Why do you need horizontal layout? What is the benefit for the browser to occupy all of the application window?

If you want to have a patient-focused, quick interface, I would make a lot more sense to me to put all of the GUI into a left-hand panel.

If you look at your current layout, there is no real need, and it looks really unnatural this wide! (below is a screenshot from your video)

You can stack search boxes one under another, or at least you can have a condensed view allowing user to switch to right-panel mode, perhaps with a reduced number of controls, but the benefit will be enormous - user can click on a series, and load it in the viewer right away, see both the DICOM browser and the image. Really - why not? From all I have seen, it is not just OHIF - this mode of navigating content is common in commercial products. It will be nice to be consistent with OHIF, but it is not about following OHIF - it is following the common practice that OHIF happened to recognize and implement.

Oct 29 '25 17:10 fedorov

I need to hardcode in CTK for which modality the rendering works

At some point, the thumbnail renderer should be made pluggable (similarly to the DisplayedFieldGenerator rules). Each registered thumbnail renderer could get a chance of inspecting a frameset, provide a confidence value, and the renderer with the highest confidence value would be used to generate the thumbnail.

Why do you need horizontal layout? What is the benefit for the browser to occupy all of the application window?

In commercial viewers, it is common to have both full-screen browser (so that the user can find the data with minimum scrolling) and a mini-browser (single column or just a few columns; that can be displayed while the image views are visible). Only having single-column browser (as in OHIF) would be a big limitation.

Having a mini-browser in Slicer is a bit more complicated than in DICOM-only viewers, because users may expect to see all data there (not just the DICOM database content but the scene content as well).

Oct 30 '25 04:10 lassoan

By the way, @Punzo the improvements look really nice!

Using half height for items that don't have a meaningful icon would be a nice touch, but the implementation cost/benefit ratio would not be very favorable. Making the thumbnail size easily adjustable (or even turning off thumbnails to maximize screen space for text data) should be sufficient for now.

Oct 30 '25 05:10 lassoan

Why do you need horizontal layout? What is the benefit for the browser to occupy all of the application window?

for my use case I need horizontal layout: per patient I can have 50-100 studies and for each study up to 40 (CT/MRI) series. I have already tried to use vertical layout and it really does not look good in this use case of mine: you can have thousands of lines and to look for the data you need, you need to scroll a lot and multi selection is a pain.

But we could easily have both vertical and horizontal layout. I agree with you that in many other cases the vertical layout is better and minimize the space used.

The good news is that I have already implemented dynamic resize for the series table (I calculate the number of columns automatically based on the width of the widget when resizing). So I would need to implement only the collapsing of the filters section (as such as you described) when resizing into a vertical layout (i.e. when the widget is just few hundreds pixels in width). This is pretty straightforward to implement.

On the other side, for the moment, I still do not have a solution to handle automatically the placing table elements to new rows for the SEG/SR modalities (ref: https://github.com/commontk/CTK/issues/1230#issuecomment-3456614658). I will think more about this. It is just an implementation issue and it is essentially an indexing issue in the series table when applying filters to the model/view architecture.

user can click on a series, and load it in the viewer right away, see both the DICOM browser and the image.

At the moment I am modifying only at CTK level and the visual DICOM browser widget is a fully working standalone window that is embedded in Slicer UI. Therefore it can be placed anywhere we want in Slicer and with any desired width/height. Currently, I simply put the visual DICOM browser widget at the same place of the traditional DICOM browser widget. If we want to change that, I have no issue on my side. But this is something we should discuss and agree with every one else, before I put dev hours in changing any layout in Slicer core. Probably the best it would be to refactor the dicom module layout and make it that the user can resize/move the dicom viewers as dockable widgets.

At some point, the thumbnail renderer should be made pluggable (similarly to the DisplayedFieldGenerator rules). Each registered thumbnail renderer could get a chance of inspecting a frameset, provide a confidence value, and the renderer with the highest confidence value would be used to generate the thumbnail.

I agree!

By the way, @Punzo the improvements look really nice!

thanks!

Using half height for items that don't have a meaningful icon would be a nice touch, but the implementation cost/benefit ratio would not be very favorable. Making the thumbnail size easily adjustable (or even turning off thumbnails to maximize screen space for text data) should be sufficient for now.

ok, I think too I need to focus on context menu and query/retrieve next week, but I would be nice to implement it and I am adding it on the to do list. We will see if the implementation will fit my dev hours remaining for this year.

thanks both for the feedback, this is very useful. I have updated my to do list (https://github.com/commontk/CTK/issues/1230#issuecomment-3402330952).

Oct 30 '25 07:10 Punzo

Please also fix the filtering in the metadata browser: https://discourse.slicer.org/t/3d-slicer-5-10-0-metadata-keyword-filter-seems-inactive/45054/2?u=lassoan it is probably an easy fix, but better if you do it to avoid merge conflicts. Thank you!

Nov 12 '25 18:11 lassoan

Please also fix the filtering in the metadata browser: https://discourse.slicer.org/t/3d-slicer-5-10-0-metadata-keyword-filter-seems-inactive/45054/2?u=lassoan it is probably an easy fix, but better if you do it to avoid merge conflicts. Thank you!

fixed, see https://discourse.slicer.org/t/3d-slicer-5-10-0-metadata-keyword-filter-seems-inactive/45054/5?u=davide_punzo

Nov 13 '25 05:11 Punzo