Multiple identical CvAnn_MLPs yield different training results

[opencv-pushbot]

Transferred from http://code.opencv.org/issues/1411

|| moeter84 - on 2011-10-07 16:25
|| Priority: Low
|| Affected: None
|| Category: ml
|| Tracker: Feature
|| Difficulty: 
|| PR: 
|| Platform: None / None

Multiple identical CvAnn_MLPs yield different training results

Hi,
when training two seperate ANN's with identical training sets and parameters, they yield completely different results. Here is what I do:

//layout is a random working layout...
m_network->create(layout,CvANN_MLP::SIGMOID_SYM,1.1,1.1);
testwork->create(layout,CvANN_MLP::SIGMOID_SYM,1.1,1.1);

//snip... fill training_hists,training_outputs,weights

//setup tr_pars...
[[CvANN]]_MLP_TrainParams tr_pars;
tr_pars.train_method=CvANN_MLP_TrainParams::RPROP;
tr_pars.term_crit.epsilon=0.001;
tr_pars.term_crit.max_iter=1000;
tr_pars.term_crit.type=CV_TERMCRIT_ITER+CV_TERMCRIT_EPS;

//run training...
qDebug()<<"train iterations:"<<
m_network->train(training_hists,training_outputs,weights,cv::Mat(),tr_pars);
qDebug()<<"test train iterations:"<<
testwork->train(training_hists,training_outputs,weights,cv::Mat(),tr_pars);


//run a simplistic test
m_network->predict(training_hists.row(0),test);
qDebug()<<test.at<float>(0)<<test.at<float>(1)<<test.at<float>(2);
testwork->predict(training_hists.row(0),test);
qDebug()<<test.at<float>(0)<<test.at<float>(1)<<test.at<float>(2);




Now, here is what I get:

train iterations: 83
test train iterations: 52

1.12094 -1.1355 -1.15789
0.997264 -1.02149 -1.15789


These results are reproducable and at each new run, they are the same. If I trigger a new training, again with the same data set and parameters, I get again a different result, even though I use new [[CvANN]]_MLP objects and destroy the old ones.

Interestingly, all later runs of train() have a lower iteration count than the first one.

History

moeter84 - on 2011-10-14 13:06

After further investigation, I guess this is okayish behaviour, as ANN weights are initialized with a Nguyen-Widrow window. Still some problems remain:

-The RNG is NOT random at all by default. It's the same for every run!
-Initialization of weights should be made user accessible.
-Why are all later runs terminating quicker than the first one?

I don't mind this "bug" any more, as I switched to FANN for Neural Networks.

Alexander Smorkalov on 2013-12-03 06:49

-   Target version set to Next Hackathon