ANTsPyNet mriSuperresolution in antspynet

Hi encountered the errors, while running image superresolution module of antspynet

        import keras as ke
        import tensorflow as tf
        import numpy as np
        image = ants.image_read(ants.get_ants_data('r16'))
        import antspynet.architectures as apa
        import antspynet.utilities as apu
        image_sr = apu.apply_super_resolution_model_to_image(image, apu.get_pretrained_network("dbpn4x"))
        Downloading data from https://ndownloader.figshare.com/files/13347617
        145252352/145251944 [==============================] - 9s 0us/step
        145260544/145251944 [==============================] - 9s 0us/step
        Traceback (most recent call last):
        File "", line 1, in
        File "/home/uname/anaconda3/lib/python3.8/site-packages/antspynet/utilities/super_resolution_utilities.py", line 279, in apply_super_resolution_model_to_image
        model = load_model(model)
        File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
        raise e.with_traceback(filtered_tb) from None
        File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init
        raise TypeError(
        TypeError: Expected trainable argument to be a boolean, but got: None

And while running the doSuperresolution.py from https://github.com/ANTsXNet/MRISuperResolution: python Scripts/doSuperResolution.py Data/Example/1097782_defaced_MPRAGE_downsampled.nii.gz 1097782_defaced_MPRAGE_superResolution.nii.gz I encountered the following errors: ValueError: Layer count mismatch when loading weights from file. Model expected 125 layers, found 43 saved layers.

Really need help to get it working. my python version python 3.8.8 tensorflow 2.8.0

Mar 08 '22 14:03 limacora45

The first example uses a 2-D image (super-resolution is 3-D) and the second example, as stated in the README, is deprecated. Try using the second image in the first example.

Mar 08 '22 15:03 ntustison

import ants import antspynet import antspynet.architectures as apa import antspynet.utilities as apu import tensorflow as tf import numpy as np t1_file="/mnt/4e9ca5bb-8e5f-43b4-850a-00a96aa2d8b4/mri_upsample/MRISuperResolution/Data/Example/1097782_defaced_MPRAGE_downsampled.nii.gz" image = ants.image_read(t1_file) image_sr=apu.apply_super_resolution_model_to_image(image, apu.get_pretrained_network("mriSuperResolution"))

Still facing error:

TypeError Traceback (most recent call last) /tmp/ipykernel_1632866/2303928518.py in 7 t1_file="/mnt/4e9ca5bb-8e5f-43b4-850a-00a96aa2d8b4/mri_upsample/MRISuperResolution/Data/Example/1097782_defaced_MPRAGE_downsampled.nii.gz" 8 image = ants.image_read(t1_file) ----> 9 image_sr=apu.apply_super_resolution_model_to_image(image, apu.get_pretrained_network("mriSuperResolution"))

~/anaconda3/lib/python3.8/site-packages/antspynet/utilities/super_resolution_utilities.py in apply_super_resolution_model_to_image(image, model, target_range, batch_size, regression_order, verbose) 277 tflite_flag = True 278 else: --> 279 model = load_model(model) 280 shape_length = len(model.input_shape) 281

~/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs) 65 except Exception as e: # pylint: disable=broad-except 66 filtered_tb = _process_traceback_frames(e.traceback) ---> 67 raise e.with_traceback(filtered_tb) from None 68 finally: 69 del filtered_tb

~/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py in init(self, trainable, name, dtype, dynamic, **kwargs) 347 (isinstance(trainable, (tf.Tensor, tf.Variable)) and 348 trainable.dtype is tf.bool)): --> 349 raise TypeError( 350 'Expected trainable argument to be a boolean, ' 351 f'but got: {trainable}')

TypeError: Expected trainable argument to be a boolean, but got: None

Mar 08 '22 15:03 limacora45

Please let me know how I can troubleshoot this to get this module working. I am using pop os 21.10 (Linux) as the operating system, in case that is necessary. I tried in ubuntu 20.04 LTS, with no luck as well. And in addition, I also received the same error when trying: image_srr=antspynet.utilities.mri_super_resolution(image)

Mar 08 '22 16:03 limacora45

I just ran it and didn't have any problem (see below). You might want to clean out your ANTsXNet cache (~/.keras/ANTsXNet) to make sure you have the correct model.

(base) ntustison@Histograms Desktop % python3
Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ants
>>> import antspynet
INFO:tensorflow:Enabling eager execution
INFO:tensorflow:Enabling v2 tensorshape
INFO:tensorflow:Enabling resource variables
INFO:tensorflow:Enabling tensor equality
INFO:tensorflow:Enabling control flow v2
>>> t1 = ants.image_read("testData2mm.nii")
>>> image_sr = antspynet.mri_super_resolution(t1, verbose=True)
2022-03-08 08:10:30.404671: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-03-08 08:10:30.924440: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
>>> ants.image_write(image_sr, "image_sr.nii.gz")

Mar 08 '22 21:03 ntustison

Hi, I tried to see whether ANTsXNet cache had files that were not correct. So I cleaned the directory (~/.keras/ANTsXNet) and re run it in two different linux distros. pop os 21.10: Python 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

import ants import antspynet t1=ants.image_read("t1.nii") image_sr = antspynet.mri_super_resolution(t1, verbose=True) Downloading data from https://ndownloader.figshare.com/files/24128618 34881536/34878800 [==============================] - 15s 0us/step 34889728/34878800 [==============================] - 15s 0us/step Traceback (most recent call last): File "", line 1, in File "/home/uname/anaconda3/lib/python3.9/site-packages/antspynet/utilities/mri_super_resolution.py", line 45, in mri_super_resolution model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False) File "/home/uname/anaconda3/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/uname/anaconda3/lib/python3.9/site-packages/keras/engine/base_layer.py", line 349, in init raise TypeError( TypeError: Expected trainable argument to be a boolean, but got: None

and the download in the process resulted in writing mriSuperResolution.h5 in the keras cache.

ubuntu 20.04 LTS: Python 3.8.8 (default, Apr 13 2021, 19:58:26) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

import ants import antspynet 2022-03-08 18:20:14.240196: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2022-03-08 18:20:14.240214: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. t1=ants.image_read("bb.nii") image_sr = antspynet.mri_super_resolution(t1, verbose=True) Downloading data from https://ndownloader.figshare.com/files/24128618 34881536/34878800 [==============================] - 3s 0us/step 34889728/34878800 [==============================] - 3s 0us/step Traceback (most recent call last): File "", line 1, in File "/home/uname/anaconda3/lib/python3.8/site-packages/antspynet/utilities/mri_super_resolution.py", line 45, in mri_super_resolution model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False) File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init raise TypeError( TypeError: Expected trainable argument to be a boolean, but got: None

so the model seems to be correct if this mriSuperResolution.h5 weight file is supposed to be written in the keras cache.

Would be nice to get it running, so please let me know what I should do. I am also new to antspynet and antspy, so your help here would be much appreciated.

Mar 09 '22 00:03 limacora45

I can't reproduce this error so there's not much I can suggest. However, I would recommend looking at the error that's being thrown in mri_super_resolution.py. It's simply loading the model weights. For some reason, I'm able to load it and you are not---I don't know why.

>>> model_and_weights_file_name = antspynet.get_pretrained_network("mriSuperResolution")
>>> model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False)

Perhaps you have an outdated version of keras/tensorflow.

Mar 09 '22 00:03 ntustison

Hi, Ntustison, First of all thank you for such quick responses, happy that after your responses, I got the urge to dig deeper. Firstly, the keras/tensorflow version that i have is the most recent ones, 2.8.0. And I created a virtual environment to install different tensorflow versions, and tried to install antspynet, and in one of the venv, i let antspynet install whatever tensorflow it wants to run with. With version 2.6.0, tf, there was a bug noted in tf also something known when I searched online, that didn't allow me to load antspynet , then, in tf version 2.7.0, the command model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False) threw this error: This version of TensorFlow Probability requires TensorFlow version >= 2.8; Detected an installation of version 2.7.0. Please upgrade TensorFlow to proceed.

So in order to run the newest antspynet edition properly tensorflow 2.8 seems important.

now the error that I mentioned earlier, was an error that comes from the model file mriSuperResolution.h5, which has to do with setting some kind of trainable argument, I am not an expert here, just starting on antspy and antspynet. which threw the error in
File "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init raise TypeError( TypeError: Expected trainable argument to be a boolean, but got: None so I opened the base_layer.py command, and commented out the boolean expectation, and re ran the command.. It worked.

Python 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

import ants import antspynet t1=ants.image_read("/media/uname/diskone/mri_ants_superresolution/MRISuperResolution/Data/Example/t1.nii") image_sr = antspynet.mri_super_resolution(t1, verbose=True) 2022-03-09 06:15:11.387914: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-03-09 06:15:11.836725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 12910 MB memory: -> device: 0, name: NVIDIA RTX A4000, pci bus id: 0000:21:00.0, compute capability: 8.6 2022-03-09 06:15:14.208739: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8302 image_sr ANTsImage (RPI) Pixel Type : float (float32) Components : 1 Dimensions : (96, 128, 128) Spacing : (2.0, 2.0, 2.0) Origin : (-91.5265, 110.0629, -142.0) Direction : [ 1. 0. 0. 0. -1. 0. 0. 0. 1.]

ants.image_write(image_sr,"/media/uname/diskone/mri_ants_superresolution/MRISuperResolution/Data/Example/t1_super.nii.gz")

It turns out "/home/uname/anaconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 349, in init raise TypeError( TypeError: Expected trainable argument to be a boolean, but got: None" is the key.

Now that this module is working for me, I do have some follow up questions, although these are not really issues in running the model, but more of the details in understanding the architectures available, and their usage, etc, but I am not sure whether this is the right platform. Its more like a discussion rather than an issue. So it would be great if you could let me know the correct platform for such queries, and discussion.

Thank you again for such quick response, the entire ANTS community do care a lot about their users.

Mar 09 '22 14:03 limacora45

Now that this module is working for me, I do have some follow up questions, although these are not really issues in running the model, but more of the details in understanding the architectures available, and their usage, etc, but I am not sure whether this is the right platform. Its more like a discussion rather than an issue. So it would be great if you could let me know the correct platform for such queries, and discussion.

Just start a separate issue.

Mar 09 '22 15:03 ntustison