tensorflow
Add additional channel to pre-trained resnet-50 model
What is the top-level directory of the model you are using: from nets.resnet_v1 import bottleneck, resnet_arg_scope
Have I written custom code: Yes, I am trying to use pre-trained Resnet50 model. However, I intend to modify the first convolution layer to be able to take a 4-channel tensor(image+binary mask), where the network weights (including the first convolutional layer 3 channels) are initialized from the pre-trained model, except for the newly added channel which are initialized randomly. Adding this feature to existing Resnet models would be really helpful for segmentation tasks.
OS Platform and Distribution: Ubuntu 16.04
TensorFlow installed from: Anaconda
TensorFlow version: 1.12.0
Bazel version: N/A
CUDA/cuDNN version: CUDA - 10.0
GPU model and memory: Nvidia Titan Xp 12GB
Exact command to reproduce: [I intend to add extra channel to the first convolutional layer, while still using the pre-trained Imagenet weights on the other channels of the first convolutional layer]
with tf.contrib.slim.arg_scope(resnet_arg_scope(batch_norm_decay=0.9, weight_decay=0.0)): with tf.variable_scope('resnet_v1_50', values=[image]) as sc: end_points_collection = sc.name + '_end_points' with slim.arg_scope([slim.conv2d, bottleneck], outputs_collections=end_points_collection): with slim.arg_scope([slim.batch_norm], is_training=is_training): net = image net = conv2d_same(net, 64, 7, stride=2, scope='conv1') net = slim.max_pool2d(net, [3, 3], stride=2, scope='pool1')
with tf.variable_scope('block1', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*64, depth_bottleneck=64, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*64, depth_bottleneck=64, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*64, depth_bottleneck=64, stride=2)
with tf.variable_scope('block2', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=1)
with tf.variable_scope('unit_4', values=[net]):
net = bottleneck(net, depth=4*128, depth_bottleneck=128, stride=2)
with tf.variable_scope('block3', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_4', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_5', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=1)
with tf.variable_scope('unit_6', values=[net]):
net = bottleneck(net, depth=4*256, depth_bottleneck=256, stride=2)
net = nonlocal_dot(net, depth=512, embed=True, softmax=True, maxpool=2, scope='nonlocal3')
with tf.variable_scope('block4', values=[net]) as sc_block:
with tf.variable_scope('unit_1', values=[net]):
net = bottleneck(net, depth=4*512, depth_bottleneck=512, stride=1)
with tf.variable_scope('unit_2', values=[net]):
net = bottleneck(net, depth=4*512, depth_bottleneck=512, stride=1)
with tf.variable_scope('unit_3', values=[net]):
net = bottleneck(net, depth=4*512, depth_bottleneck=512, stride=2)
net = tf.reduce_mean(net, [1, 2], name='pool5', keep_dims=False)
endpts = slim.utils.convert_collection_to_dict(
end_points_collection)
endpts['model_output'] = endpts['global_pool'] = net
Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks. What is the top-level directory of the model you are using Have I written custom code OS Platform and Distribution TensorFlow installed from TensorFlow version Bazel version CUDA/cuDNN version GPU model and memory Exact command to reproduce
