Improve summaries and visualizations of results with modularized decision tree

[handwerkerd]

Write code to automatically create a flow chart or other summary/visualization for a decision tree. This can also be used to make a run-specific visualization that shows how components were reclassified at each node of the decision tree.

Summary

The decision tree modularization (#756) is getting close to done. The "Decision Tree" is the series of steps through which each ICA component is classified as accepted or rejected. A nice feature of this work is a lot of useful information about what happened during processing is logged in structured formats. This information can be visualized and summarized in ways that will help users better understand what is happening and better understand their own data. This can be done by someone who enjoys writing code for interactive visualizations and flow charts even if they don't know much about the internal workings of tedana.

Additional Detail

Flow chart for decision tree

• #355 has an extended discussion on different ways to visualize the decision tree in flow charts. All of these were semi-manually visualized.
• ./tedana/resources/decision_trees contains JSON files for each decision tree that is distributed with tedana. There are a series of elements that detail each step in the process and these can be used to generate a flow chart
• After tedana is run on a dataset, a file called desc-ICA_decision_tree.json is saved with the rest of the program's outputts. That file contains the same decision tree that was ran along with an output key for each step in the process. Information on what how many component classifications changed and what they changed to is stored there. Other values, such as calculations for various thresholds are also stored. It should be possible to generate a flow charts that highlights exactly which steps affected component classifications.
• Another potentially useful output is desc-ICA_status_table.tsv which has a row for each component and a column for each step (node) in the decision process. Each components classification after each node is stored there.

Interactive report visualizations

• When tedana is run, the output includes an interactive visualization of the results in tedana_report.html This currently includes the components with there kappa values, rho values, variance explained, final classifications, spatial maps, and time series. This information is displayed across several subplots.
• Currently accepted components are green and rejected components are red.
• We will now be saving classification tags for each component and each component could have more than one tag. It would be useful to be able to highlight subsets of components using these classification tags.
• Additional information, such as a mini summary of when components changed their classifications and thresholding metrics that were calculated across components could also be visualized or summarized in this report.

Next Steps

• If you are interested in working on this, and haven't previously contributed to tedana, let @handwerkerd know so that he can provide additional guidance
• A bit more information on how things are named and stored is in ./tedana/docs/building_decision_trees.rst