MDT issues in NODDI maps using MDT fitting

I managed to install mdt via docker on a Centos7 server. I tried to fit this model on HCP data and fitting is very fast (about 6 minutes per subject). However, something goes wrong since NODDI maps look correct just for a small subset of slices on the left of the image, and the rest appear all white. I am attaching NDI for one subject as an example. How could I solve this issue? thanks in advance, Rosella IMG-20231120-WA0005

Mar 08 '24 14:03 rosella1234

Hi Rosella, this sometimes happens when there are too many voxels loaded at once into the GPU. You could add a file to your system in ~/.mdt/<version>/mdt.conf with the content:

processing_strategies:
    optimization:
        max_nmr_voxels: 5000

    sampling:
        max_nmr_voxels: 1000

Or any sufficiently lower number. This should reduce the number of voxels in one batch and as such get rid of the problem.

Sorry for the slow reply, I had to take care of my son.

Best,

Robbert

Mar 26 '24 14:03 robbert-harms

Hi, thanks for your suggestion. I managed to install MDT in my centos server through a virtual environment, and reduced the number of voxels as you suggested. However, it takes ages getting stuck in processing for the first subject and I had to stop it. Do you have any suggestion? How long should it take to process one subject? In our centos server we do have gpu. thanks Rosella

Apr 03 '24 14:04 rosella1234

Hi Rosella,

It looks like I might be running into a similar situation to yours. Are you getting stuck in the “optimizing” stage?

Best Wishes, Elijah

On 3 Apr 2024, at 15:37, rosella1234 @.***> wrote:

Hi, thanks for your suggestion. I managed to install MDT in my centos server through a virtual environment, and reduced the number of voxels as you suggested. However, it takes ages getting stuck in processing for the first subject. I want to try via docker (I remember it took less even if it didn't work as I pointed out in current issue). However, installing via docker, I can't find a ~/.mdt//mdt.conf file where to reduce the number of voxels. Do you hve any suggestion? How long should it take to process one subject? In our centos server we do have gpu. thanks Rosella

— Reply to this email directly, view it on GitHubhttps://github.com/robbert-harms/MDT/issues/60#issuecomment-2034807686, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE57L2WSKTA226WV3OWVOFLY3QH4BAVCNFSM6AAAAABEM42UMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZUHAYDONRYGY. You are receiving this because you are subscribed to this thread.Message ID: @.***>

Apr 03 '24 14:04 ElijahMak

I have decreased the number of voxels in the conf file in MDT installed via docker. And this is what I get: The result is better than before but still suboptimal: should I further decrease the number of voxels in the file? thanks Rosella

Apr 03 '24 15:04 rosella1234

Hi Rosella, It looks like I might be running into a similar situation to yours. Are you getting stuck in the “optimizing” stage? Best Wishes, Elijah

Yes exactely

Apr 03 '24 15:04 rosella1234

Hi Rosella, Elijah, this is a known problem on Nvidia GPU's. It unfortunately got worse over time, Nvidia does not actively support the open source standard OpenCL but instead favor their proprietary CUDA solution.

You could try with another optimization routine, i.e. NMSimple or Subplex. Sometimes this compiles better and give better results. You could also try to install POCL and run it on the CPU. It will not give you the speed benefits though.

Being stuck in the optimization phase typically means that the Nvidia OpenCL compiler had trouble compiling the kernels. You could try to see if you can update your NVidia driver, perhaps a new version will work.

There is not much more I can do. I have some plans for an MDT v2.0 which would do this fundamentally different, but it is very difficult to get funding for developing open source scientific software.

Apr 04 '24 10:04 robbert-harms

Thank you very much! I will try with your suggestions. So you think even further reducing the number of voxels in the conf file will not solve the problem with the current configuration via docker?

Apr 11 '24 08:04 rosella1234

Well, it could solve the problem, but the time to process will drastically increase. Lately it has been a lot of hit or miss with nvidia gpu's. Unfortunately.

Apr 24 '24 08:04 robbert-harms

Many thanks Robbert, I fully understand the constraints and appreciate all your work on the MDT toolbox. Unfortunately, we could not get MDT to work on our HPC eventually and had to switch over to AMICO. Thankfully, the results are very similar too :)

Will be looking forward to the release of MDT 2.0 in future!

Best Wishes, Elijah

On 24 Apr 2024, at 09:53, Robbert Harms, PhD @.***> wrote:

Well, it could solve the problem, but the time to process will drastically increase. Lately it has been a lot of hit or miss with nvidia gpu's. Unfortunately.

— Reply to this email directly, view it on GitHubhttps://github.com/robbert-harms/MDT/issues/60#issuecomment-2074434822, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE57L2RIF5QZ6WJETHRDLLTY65XJJAVCNFSM6AAAAABEM42UMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZUGQZTIOBSGI. You are receiving this because you commented.Message ID: @.***>

Apr 24 '24 09:04 ElijahMak

Very understandable! Wishing you all the best, and perhaps see you again with MDT v2.0.

Apr 24 '24 09:04 robbert-harms