second infer-pass output same as first infer-pass

[sysuxiaoming]

Hi, all

I found a very strange bug.

I infer the model densenet-121 with mkldnn engine with intel-caffe , but here is what happen: after i init network, the first infer pass is correct, but the following infer pass outputs the same result as the first infer pass, no matter what i input.

p.s. the densenet mode comes from https://github.com/shicai/DenseNet-Caffe

For reproduce my bug. here is what you need to do

step 1: get the prototxt from https://raw.githubusercontent.com/shicai/DenseNet-Caffe/master/DenseNet_121.prototxt

step 2: remove line 54"ceil_mode: false" in the prototxt since intel caffe not supported.

step 3: get the caffemodel from https://drive.google.com/file/d/0B7ubpZO7HnlCcHlfNmJkU2VPelE/view

step 4: get the main code test_bug.cpp

#define CPU_ONLY
#include <caffe/caffe.hpp>
#include <iostream>
using namespace caffe;
using namespace std;

int main(int argc, char* argv[]) {
    Caffe::set_mode(Caffe::CPU);
    const char model_file[] = "DenseNet_121.prototxt";
    const char weight_file[] = "DenseNet_121.caffemodel";
    const char* engine = argv[1]; // "MKLDNN" or "MKL2017" or "CAFFE"
    Net<float> * net_ = new Net<float>(model_file, TEST, 0, NULL, NULL, engine);
    net_->CopyTrainedLayersFrom(weight_file);
    Blob<float>* input_layer = net_->input_blobs()[0];
    input_layer->Reshape(1, 3, 224, 224);
    int pix_count = 1 * 3 * 224 * 224;
    float* input_data = input_layer->mutable_cpu_data();
    const float* result_data = net_->output_blobs()[0]->cpu_data();
    
    cout << "*******first infer pass*********" << endl;
    for (int i = 0; i < pix_count; i++) {
        input_data[i] = i % 255;
    }
    net_->Forward();

    //when i comment this code, i get zero output with mkldnn or mkl2017.
    //it is another strange bug.
    const float* temp_result_data = net_->blob_by_name("fc6")->cpu_data();
    //temp_result_data is the same as result_data,you can check by uncomment the follow code
    /*
    cout << static_cast<const void*>(temp_result_data) << endl;
    cout << static_cast<const void*>(result_data) << endl;
    */
    
    for (int i = 0; i < 10; i++) {
        cout << result_data[i] << " ";
    }

    cout << endl << "*******second infer pass*********" << endl;
    for (int i = 0; i < pix_count; i++) {
        input_data[i] = 255 - i % 255;
    }
    net_->Forward();
    for (int i = 0; i < 10; i++) {
        cout << result_data[i] << " ";
    }
    cout << endl;
}

step 5: compile by the follow command

g++ -std=c++11 -o test_bug test_bug.cpp -I"/usr/local/include" -I"/caffe/include" -L"/caffe/build/lib" -lboost_system -lcaffe -D_REENTRANT

step 6: test by yourself

./test_bug MKLDNN
./test_bug MKL2017
./test_bug CAFFE

you will get a very strange result. when you use MKLDNN or MKL2017 engine it work correct in the first infer pass. but no matter what you input, the follow infer pass have the same result as the first pass. when you use CAFFE engine ,it always work well.