flare icon indicating copy to clipboard operation
flare copied to clipboard

mean absolute error too big. How to improve it?

Open kitpeng11 opened this issue 5 years ago • 5 comments

The error is too big after training GP model from aimd. How to improve? Thanks

mean absolute error: 241.92 meV/A mean absolute dft component: 883.22 meV/A mae per species type O mae: 268.13 meV/A type Li mae: 116.08 meV/A type P mae: 435.55 meV/A type N mae: 326.57 meV/A

kitpeng11 avatar Sep 08 '20 09:09 kitpeng11

Hi kitpeng11,

It's possible that your system has many-body interactions that a 2+3-body potential can't capture. I recommend computing the learning curve (MAE on an independent test set as a function of training set size) to confirm that the error won't drop with additional training data.

Best, Jon

jonpvandermause avatar Sep 10 '20 14:09 jonpvandermause

Just to check, could you also report the cutoff and likelihood gradient you're seeing?

jonpvandermause avatar Sep 10 '20 15:09 jonpvandermause

The reporting result is not calculated by 2+3-body model ( which takes really long). The reporting result is from 2 body. The cutoff is 5 and the likelihood gradient various. Do you mean the last one? Thanks a lot for your reply.

kitpeng11 avatar Sep 12 '20 12:09 kitpeng11

Isn’t a 2-body kernel supposed to give good performance only for some special cases?

mkrompiec avatar Sep 21 '20 19:09 mkrompiec

Hi kitpeng11,

As mkrompiec mentioned, pure 2-body potentials will only give acceptable performance in very simple settings, e.g. simple crystals at low temperature; most materials require at least a 3-body contribution.

Have you tried the 2+3-body kernel yet? What kind of MAE do you see for that?

Best, Jon

jonpvandermause avatar Oct 01 '20 13:10 jonpvandermause

Close stale issue

YuuuXie avatar Dec 04 '22 16:12 YuuuXie