Example to train synthmorph

[jackyko1991]

I find the synthmorph training script in ./scripts/tf/train_synthmorph.py

From my understanding, synthmorph requires pre-training of contrast pattern model, then using transfer learning technique to adopt to medical image dataset.

Any instructions or examples will be given to generate the initial training data, scripts for transfer learning stage?

I am currently using DIR-LAB and COPDgene dataset for lung registration. I am trying to evaluate if synthmorph is also applicable in chest CT data. Any help would be appreciated.

[adalca]

From my understanding, synthmorph requires pre-training of contrast pattern model, then using transfer learning technique to adopt to medical image dataset.

@jackyko1991 no we don't do any pre-training or transfer learning. Please see a demo here: https://colab.research.google.com/drive/1n3ehxvrn33imAKGMlNga4IY14trP5_Um?usp=sharing

@mu40 can give more help

[mu40]

Yes, we do not. While we did not include CT in our evaluation either, I suspect that normalizing the intensities of your CT data may be helpful, or, alternatively, adapting the synthesis to include the intensity distribution of CT scans, e.g. by playing with the gamma augmentation. The demo is a good place to start.

[jackyko1991]

I have completed the traning on synthmorph and checked with hypermorph. I am curious if the hypernetwork need to be combined with synthmorph for an automated hyperparameter tuning.

One question about hypermorph. During network prediction I need to provide the lambda value as input

# https://github.com/voxelmorph/voxelmorph/blob/dev/scripts/tf/train_hypermorph.py
# sweep across 20 values of lambda
for i, hyp in enumerate(np.linspace(0, 1, 20)):
    hyp = np.array([[hyp]], dtype='float32')  # reformat hyperparam
    img = model.predict([moving, fixed, hyp])[0].squeeze()
    moved.append(img)

Does this refers that we need to benchmark the registration accuracy independently but not direclty optimizing the hyperparameter?

When I use segmentation labels that are highly unbalanced with volume in deep learning, I often use generalized DICE loss/ weighted cross entroopy. Have you ever tried to tune the class weights in hypermorph for unbalanced labels?