spark-libFM icon indicating copy to clipboard operation
spark-libFM copied to clipboard

I got an error when run FMWithLBFGS.train

Open xueyuan1990 opened this issue 5 years ago • 1 comments

I got an error when run FMWithLBFGS.train. The message is

19/03/18 16:02:48 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
19/03/18 16:02:48 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
19/03/18 16:02:58 INFO StrongWolfeLineSearch: Line search t: 0.11782555073452997 fval: 0.18211103495544895 rhs: 0.2606692174807194 cdd: 1.0498989669795253
19/03/18 16:02:58 INFO LBFGS: Step Size: 0.1178
19/03/18 16:02:58 INFO LBFGS: Val and Grad Norm: 0.182111 (rel: 0.302) 0.509790
19/03/18 16:03:02 INFO StrongWolfeLineSearch: Line search t: 0.36638861081271834 fval: 0.18211103495544895 rhs: 0.18211004643573764 cdd: -0.0115736658307402
19/03/18 16:03:05 INFO StrongWolfeLineSearch: Line search t: 0.09905059408456629 fval: 0.18211103495544895 rhs: 0.18211076771607038 cdd: -0.02229934335451164
19/03/18 16:03:06 INFO StrongWolfeLineSearch: Line search t: 0.02245692390442945 fval: 0.18211103495544895 rhs: 0.1821109743664697 cdd: -0.02588247744386086
19/03/18 16:03:08 INFO StrongWolfeLineSearch: Line search t: 0.004822954619910672 fval: 0.18211103495544895 rhs: 0.18211102194307474 cdd: -0.02674252026232241
19/03/18 16:03:10 INFO StrongWolfeLineSearch: Line search t: 0.0010227603171640213 fval: 0.18211103495544895 rhs: 0.18211103219603256 cdd: -0.026929624963460522
19/03/18 16:03:12 INFO StrongWolfeLineSearch: Line search t: 2.1629420637255935E-4 fval: 0.18211103495544895 rhs: 0.18211103437188528 cdd: -0.026969412562859015
19/03/18 16:03:15 INFO StrongWolfeLineSearch: Line search t: 4.5715477240673296E-5 fval: 0.18211103495544895 rhs: 0.1821110348321082 cdd: -0.02697783181839336
19/03/18 16:03:19 INFO StrongWolfeLineSearch: Line search t: 9.661135718329927E-6 fval: 0.18211103495544895 rhs: 0.18211103492938313 cdd: -0.026979611515011322
19/03/18 16:03:20 INFO StrongWolfeLineSearch: Line search t: 2.041652436213748E-6 fval: 0.18211103495544895 rhs: 0.18211103494994055 cdd: -0.026979987631432584
19/03/18 16:03:23 INFO StrongWolfeLineSearch: Line search t: 4.3145256182019533E-7 fval: 0.18211103495544895 rhs: 0.18211103495428488 cdd: -0.02698006711517943
19/03/18 16:03:23 ERROR LBFGS: Failure! Resetting history: breeze.optimize.FirstOrderException: Line search zoom failed
19/03/18 16:03:28 INFO StrongWolfeLineSearch: Line search t: 0.2313204034196764 fval: 0.18211103495544895 rhs: 0.18210502325583827 cdd: 0.15801565968824438
19/03/18 16:03:29 INFO StrongWolfeLineSearch: Line search t: 0.10214095643616886 fval: 0.18211103495544895 rhs: 0.1821083804522622 cdd: 0.047316513760600414
19/03/18 16:03:29 INFO StrongWolfeLineSearch: Line search t: 0.03727126336829138 fval: 0.18211103495544895 rhs: 0.18211006632649893 cdd: -0.11267143347262713
19/03/18 16:03:35 INFO StrongWolfeLineSearch: Line search t: 0.010054503272280822 fval: 0.18211103495544895 rhs: 0.18211077365271683 cdd: -0.2152298221473921
19/03/18 16:03:39 INFO StrongWolfeLineSearch: Line search t: 0.002278000948200439 fval: 0.18211103495544895 rhs: 0.1821109757533327 cdd: -0.2494210814426842
19/03/18 16:03:41 INFO StrongWolfeLineSearch: Line search t: 4.891533182012405E-4 fval: 0.18211103495544895 rhs: 0.18211102224302597 cdd: -0.2576215252050019
19/03/18 16:03:42 INFO StrongWolfeLineSearch: Line search t: 1.0372658433615846E-4 fval: 0.18211103495544895 rhs: 0.18211103225973746 cdd: -0.2594052144599783
19/03/18 16:03:49 INFO StrongWolfeLineSearch: Line search t: 2.1936016937586954E-5 fval: 0.18211103495544895 rhs: 0.182111034385362 cdd: -0.25978449808886667
19/03/18 16:03:51 INFO StrongWolfeLineSearch: Line search t: 4.636341753405551E-6 fval: 0.18211103495544895 rhs: 0.1821110348349568 cdd: -0.25986475569277256
19/03/18 16:03:52 INFO StrongWolfeLineSearch: Line search t: 9.798062777729317E-7 fval: 0.18211103495544895 rhs: 0.18211103492998512 cdd: -0.2598817208409102
19/03/18 16:03:52 ERROR LBFGS: Failure again! Giving up and returning. Maybe the objective is just poorly behaved?
19/03/18 16:03:52 INFO LBFGS: Converged because line search failed!

But there's no error when I use FMWithSGD.train with the same data set. I don't know why? Thanks for your help!

xueyuan1990 avatar Mar 18 '19 08:03 xueyuan1990

the lbfgs is low level optimized

fooSynaptic avatar Mar 25 '20 04:03 fooSynaptic