js-regression
js-regression copied to clipboard
chen0040
The regression learns NaN weights with the example code.
However, if I try a simple model like y= mx+c, then it gives some weights but never gets the y-intercept right. It's always close to zero. I did try playing around with parameters/ increasing data but it still gives the same fit without y-intercept. Seems like there is some bug in the gradient descent method.
Here is the code I tried
Sorry for the mess in the above comment. Here is the code - https://pastebin.com/6BmGhcTV
Hi @gursimar Many thanks for notifying me the issue and sharing with me your code, I will look into this and get back to you :)
Thanks, I think the issue is with the gradient function. You are using a numerical method but I suppose we can use a closed form solution. Right now, I'm trying to do that but I'm using math library to do this.
Many thanks @gursimar , i think you are right. i believed the gradient descent method is very sensitive the values of the learning rate alpha and the regularization lambda. I played with a few small values of alpha and lambda using your code sample, it did converge with NaN issue disappear (if you set a sufficiently small alpha value) but behavior is quite sensitive to the value of alpha indeed, i will consider the closed form.
Hi @gursimar after some close examination on the linear regression source code, i notice that the gradient calculation has a bug which i have now addressed. I have also added a number html demo on the linear regression:
https://rawgit.com/chen0040/js-regression/master/example-regression-3.html https://rawgit.com/chen0040/js-regression/master/example-regression-2.html https://rawgit.com/chen0040/js-regression/master/example-regression.html
The y-intercept is still not as effective, subject to the learning rate and regularization.
Hi @chen0040, I tried the regression with random data like this;
var jsregression = require('js-regression');
var data = [];
for (var i = 0; i<200; i++) {
data.push([i,Math.random()*10*i+20]);
}
var regression = new jsregression.LinearRegression();
var model = regression.fit(data);
console.log(model);
output is the follwing:
{ theta: [ NaN, NaN ], dim: 2, cost: NaN, config: { alpha: 0.001, lambda: 0, iterations: 1000 } }
when I change the configuration values for alpha and lambda a bit, I do get different results, but nothing remotely similar to the equation used to generate the data.
am I missing something here or is there still a bug present?
