using 'linear' output seems to result in divergence/ NaN's very quickly instead of finding good weights

[dinosg]

Hello, Using Matlab's house_dataset from its neural networks toolbox as a test example for a linear rather than logistical problem, seems to crash the back propagation and training algorithms. Printing intermediate results out shows the weights diverge to astronomical numbers very quickly.

The house_dataset example has training data that's 13 x 506, and 'label' data that's 1 x 506 because the 'label' is a real number not a logical.

Example code below shows rapid divergence of nn.W and nn.a's

clear load house_dataset ;% use test house dataset, 13 variables in houseInputs, % for house values, houseTargets, with 506 different samples. so % houseInputs is 13 x 506, houseTargets is 1x506.

inputs = houseInputs(:,1:400)'; %select training data and transpose to deeplearntoolbox format targets = houseTargets(1:400)'; %from Matlab format

rand('state',0); nn = nnsetup([size(inputs,2) 100 1]); nn.output='linear'; %this is a linear problem opts.numepochs = 1; % Number of full sweeps through data opts.batchsize = 1; % Take a mean gradient step over this many samples [nn, L] = nntrain(nn, inputs, targets, opts);

trainpredict= houseInputs(:,401:420)'; targetpredict = houseTargets(1,401:420)'; nullpredict = zeros(1,20)'; nn.testing = 1;

nn = nnff(nn, trainpredict, nullpredict); nn.testing = 0;

[dinosg]

OK, I reduced the learning weight to 0.01, that helped and I get a fit. However using the dbn routines in addition to the nn training doesn't help any (the dbn routines do reduce error by a factor of 2 on the NIST handwriting recognition database). Any idea why dbn doesn't work as well with linear data or am I missing something?

[Duanexiao]

Hello,may I ask you some questions?firstly,I know you have used the DBN+NN for regression prediction,Can you share me some big data (the feature and the number of dataset are both big enough for using DBN+NN)？I have downloaded the blogDatatrain dataset for UCI,but when I test DBN+NN for regression prediction on it,the outcome is so bad, and the outputs are all the same! Secondly,when I use the Matlab's house_dataset ,and split it into three parts,train_x,val_x,test_x.using the nn for regression prediction,it throws an error in nnsetup,that caused by the inner nn toolbox in Matlab.I change the nnsetup to nnsetups,but another problem occurs,none-existence field batchsize in batchsize = opts.batchsize(in nntrain.m),why? Thank you in advance!

[sundw2014]

Maybe, it's 'learning rate', but not 'learning weight'. That worked! Thank you.