WiP Convergence on the use-cases analysis

[yarikoptic]

Original/conventional Heudiconv operation

• Overall a two stage procedure

  1. prepare the conversion table for an accession using a heuristic (need -c none to run in this mode)
     • in the output directory creates .heudiconv/<sub>[/<ses>]/info/ where it places:
       • copies the heuristic file
       • copies the heudiconv.py
       • creates .auto and .edit conversion table
  2. [optional] edit the NAME.edit.txt to tune up the conversion
  3. apply the conversion table to the data
     • (@yarikoptic thinks that) it is the same command line invocation as in prepare step. (@mgxd thinks that it is triggered by specifying a converter, e.g. dcm2niix)
     • Q: copies the heuristic file again? probably should be avoided

• If the conversion isn't proper (incomplete, erroneous),

  1. edit the conversion table and re apply
     • Q: how does it remove "erroneous" files (if at all?) -- probably doesn't
     • overwrite already existing files (possibly requires already use of --overwrite file)
  2. edit the heuristic and re apply
     • In theory (not checked if implemented ATM) if a heuristic is changed then prepare should be redone, i.e. conversion table should be recreated (see https://github.com/nipy/heudiconv/issues/84#issuecomment-330048609)

@mgxd workflow for multiple subjects, study specific

1. take a heuristic (e.g. convertall, @mgxd -- no -f it would default convertall)
2. run prepare (-c None)
3. looking at .info file, make a study heuristic
4. remove previous prepare result and now do the two stage procedure for the subject
5. for any new subject just redo 1+3 in one command (like below for ReproIn use case)
   • if anything screws up, just remove subject conversion and redo

ReproIn/Dartmouth DBIC approach/needs

heudiconv -f ~/heudiconv/heuristics/dbic_bids.py -c dcm2niix -o /inbox/BIDS --bids --datalad PATHTODICOMS

So it all worked collapsing both stages because of use of -c dcm2niix which instruments the converter and thus causes the conversion right away:

• if the conversion existed for that subject/session, it would be reused (instead of consulting the heuristic). caused some confusion see #84

• that is why --overwrite was introduced to make it so by default no file gets modified

• PATHTODICOMS typically contains a single accession (acquisition) for a subject in some session

• we run it once and it does both stages

Items

be able to provide --bids mode of operation by heuristic

--overwrite should not be needed in any of above usecases

If any file gets overridden (besides modifying participants.tsv, _scans.tsv, etc) -- that is a sign of trouble, and check should be done and --overwrite should be used with caution

conflated 2 stage procedure behavior whenever converter specified

As in #84 it is might be confusing whenever you run "full conversion" but it reuses previous conversion table.

[yarikoptic]

@mgxd could you please add sample heudiconv invocations you use (e.g. either you use subj template etc)?

[mgxd]

when working with a new dataset:

I generally use the subj template command

heudiconv -d /path/to/data/{subject}/*/*.dcm -f convertall.py -c none -o /output -s $subj

which generates the /output/.heudiconv/${subj}/info/dicominfo.tsv file I parse and generate a specific heuristic for the dataset (if fixed parameters)

then, when applying

heudiconv -d /path/to/data/{subject}/*/*.dcm -f new_heuristic.py -c dcm2niix -o /output -s $subj -b --minmeta

[yarikoptic]

#129 largely addressed it but I think we should make modes more explicit may be by changing cmdline API to first have a command