Consistency in clustering scripts for subject folder creation

[nsomes]

While the rest of the scripts in the clustering pipeline automatically create a subject folder in the specified directory, the script wm_cluster_remove_outliers.py does not. If you don't know this going into using it, it can lead you overwriting the same folder when running the script for multiple subjects in series.

Somewhat related, but a bit different, the script wm_append_cluster.py automatically names the appended cluster "cluster_appended_" followed by the numbers of the clusters that were appended. When you are organizing your output by tract, this leads to all of the subjects for one tract being named the same thing and over-writing each other. Maybe the automated naming could include the case name in it, or you could be allowed to input your own name since this is often not the naming convention I want to go with anyways?

[ljod]

Thanks Nate!

[zhangfanmark]

@nsomes Thanks!

About your first question about wm_cluster_remove_outliers.py, this script takes the folder of fiber clusters of one subject as input, and considers the folder name as subject ID. Then, it actually creates a new sub folder 'subjectID_outlier_removed' under the output dir.

https://github.com/SlicerDMRI/whitematteranalysis/blob/master/bin/wm_cluster_remove_outliers.py#L60-L64

Could you provide details how the results overwrite your previous files?

Fan

[zhangfanmark]

@nsomes

For your second question about wm_append_cluster.py, our recent updates allow users to provide a file name using '-o'. If assigned, it will overwrite the default filename 'cluster_appended_XX.vtp'.

https://github.com/SlicerDMRI/whitematteranalysis/blob/master/bin/wm_append_cluster.py#L20-L22

Please check if the version you used is the latest.

Regards, Fan

[nsomes]

Hi Fan,

Thanks for the info It looks like I might have been using an older version of the append script. In terms of the outlier removal script, it's very possible that I was using an older version there too (which I will look into updating if that's the case), but when I ran it, the output folder was called "_outlier_removed" without putting the input folder name in front of it, which made it overwrite itself as it did each subject.

Best, Nate

---

From: Fan Zhang [[email protected]] Sent: Monday, August 07, 2017 6:41 AM To: SlicerDMRI/whitematteranalysis Cc: Somes, Nathaniel G.; Mention Subject: Re: [SlicerDMRI/whitematteranalysis] Consistency in clustering scripts for subject folder creation (#61)

@nsomeshttps://github.com/nsomes

For your second question about wm_append_cluster.py, our recent updates allow users to provide a file name using '-o'. If assigned, it will overwrite the default filename 'cluster_appended_XX.vtp'.

https://github.com/SlicerDMRI/whitematteranalysis/blob/master/bin/wm_append_cluster.py#L20-L22

Please check if the version you used is the latest.

Regards, Fan

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/SlicerDMRI/whitematteranalysis/issues/61#issuecomment-320630828, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYzz7C6VC-pRgjTkIp3OvAS8iUMJPpXDks5sVunPgaJpZM4OuH8w.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[zhangfanmark]

Just one possible reason, perhaps you used '.' (current folder) as input, so the script failed to identify the folder name. If you give the full path of the folder, I think it should work as expected.

Regards, Fan

[ljod]

Hi Nate. Please re-test with the latest version. Then give us an update. We want the interface to be consistent across all the scripts so that you should not need to do anything different like -o to get the expected behavior. Also I agree there should be no possible way to get an output folder called _outlier_removed so we should fix that too if it still happens. Thanks!

[nsomes]

To clarify the issue I brought up with wm_cluster_remove_outliers.py, I ran the script as follows:

python /data/pnl/soft/whitematteranalysis-2017-05-23/bin/wm_cluster_remove_outliers.py Clustered_subjects/case1_reg/ FiberClusterAtlas/ Outliers_removed/

I was inputting the output directory of clustering an individual case from the atlas. I also gave it the directory containing the atlas and the output was an empty directory to put the output directory of each case for outlier removal.

I was expecting that the output directory would be called "case1_reg_outliers_removed/". However, it ended up outputting a directory called "_outliers_removed" instead that did not contain the subject ID. When I did this with multiple cases, it overwrote the _outliers_removed directory for each case because of this. I'm not sure what could be causing this, because like Fan said this should only happen when the input directory is ".". I may be doing something silly here and I apologize if that ends up being the case.