Terrible evaluate results on modified vgg deeplab_v2 structure

[ethanygao]

Hello DrSleep,

I am not sure whether it is appropriate to ask here, but the tensorflow-deeplab-resnet repo did not contain vgg backbone currently. Therefore, I was trying to modify your lfov codes to use vgg deeplab_v2 structure (i.e. no avg_pool5, and ASPP on fc6 with 4 different atrous holes and sum them up at fc8). Below is the codes I modified (also I modified the network structure "net_skeleton.ckpt" in util folder):

@@ -112,7 +112,7 @@ class DeepLabLFOVModel(object):
         v_idx = 0 # Index variable.
         
         # Last block is the classification layer.
-        for b_idx in xrange(len(dilations) - 1):
+        for b_idx in xrange(len(dilations) - 3):
             for l_idx, dilation in enumerate(dilations[b_idx]):
                 w = self.variables[v_idx * 2]
                 b = self.variables[v_idx * 2 + 1]
@@ -138,18 +138,34 @@ class DeepLabLFOVModel(object):
                                          ksize=[1, ks, ks, 1],
                                          strides=[1, 1, 1, 1],
                                          padding='SAME')
-                current = tf.nn.avg_pool(current, 
-                                         ksize=[1, ks, ks, 1],
-                                         strides=[1, 1, 1, 1],
-                                         padding='SAME')
-            elif b_idx <= 6:
-                current = tf.nn.dropout(current, keep_prob=keep_prob)
+
+        current_before_fc = current
+        fc_layers = []
+        for i in range(len(dilations[5])):   # dilations[5] = [6, 12, 18, 24]
+            w = self.variables[v_idx * 2]
+            b = self.variables[v_idx * 2 + 1]
+            v_idx += 1
+            conv = tf.nn.atrous_conv2d(current_before_fc, w, dilations[5][i], padding='SAME')
+            current = tf.nn.relu(tf.nn.bias_add(conv, b))
+            current = tf.nn.dropout(current, keep_prob=keep_prob)
+
+            w = self.variables[v_idx * 2]
+            b = self.variables[v_idx * 2 + 1]
+            v_idx += 1
+            conv = tf.nn.conv2d(current, w, strides=[1, 1, 1, 1], padding='SAME')
+            current = tf.nn.relu(tf.nn.bias_add(conv, b))
+            current = tf.nn.dropout(current, keep_prob=keep_prob)
+
+            w = self.variables[v_idx * 2]
+            b = self.variables[v_idx * 2 + 1]
+            v_idx += 1
+            conv = tf.nn.conv2d(current, w, strides=[1, 1, 1, 1], padding='SAME')
+            current = tf.nn.bias_add(conv, b)
+
+            fc_layers.append(current)
+
+        current = tf.add_n(fc_layers)
         
-        # Classification layer; no ReLU.
-        w = self.variables[v_idx * 2]
-        b = self.variables[v_idx * 2 + 1]
-        conv = tf.nn.conv2d(current, w, strides=[1, 1, 1, 1], padding='SAME')
-        current = tf.nn.bias_add(conv, b)
 
         return current

It is able to run by loading the trained model provided by http://liangchiehchen.com/projects/released/deeplab_aspp_vgg16/prototxt_and_model.zip. However, the evaluation is terrible, the mean IoU is only 0.5%. (It works good on the original lfov vgg version)

Would you please kindly give me some hints about this issue so that I can create a pull request to include the vgg deeplab_v2 structure?

Thanks!