leela-chess icon indicating copy to clipboard operation
leela-chess copied to clipboard

Leela grossly over estimates KRKB

Open vdbergh opened this issue 6 years ago • 2 comments

Leela evaluates KRKB as +6 even though it is usually theoretically draw. As a result she throws away won endgames by doing bad conversions.

It is interesting to speculate why this is the case. It must be because this is an exception to the rule that being an exchange ahead is a huge advantage in the endgame, all other things being equal.

One may also wonder if she will ever learn to evaluate KRKB correctly with the current training methodology.

vdbergh avatar May 02 '18 06:05 vdbergh

It is interesting to speculate why this is the case. It must be because this is an exception to the rule that being an exchange ahead is a huge advantage in the endgame, all other things being equal.

That's not quite true, because IIRC plain KR vs. KN is evaluated as drawish by LCZero. So the reason is really unclear. BTW I also have seen those KR vs. KB endgames and wanted to write about it, but you were faster.

rwbc avatar May 02 '18 07:05 rwbc

I was wrong for the evaluation KR vs. KN. LCZero evaluates that also as winning with +4 - +6. BTW even RK vs. BPK is evaluated as winning!

In my current test bed LCZero already has 9/600 missevaluated endings of RK vs. BK or BPK and 3/600 with missevaluated RK vs. NK or NPK. Of course all ended as 50 moves draws.

 55 LCZero_07ID181  Danasah_70      (B30) 2018.04.25 =-= (116) [r:b]
349 LCZero_07ID150  Monolith_04-64  (D20) 2018.04.27 =-= (117) [r:b]
355 LCZero_07ID150  Danasah_70      (D00) 2018.04.27 =-= (175) [r:b]
359 LCZero_07ID150  Tucano_400-64   (D07) 2018.04.27 =-= (117) [r:b]
429 LCZero_07ID150  Monolith_04-64  (A45) 2018.04.28 =-= (118) [r:b]
257 LCZero_07ID181  Glaurung_201-64 (A05) 2018.04.26 =-= (151) [r:b1]
 99 LCZero_07ID181  Tucano_400-64   (C45) 2018.04.25 =-= (136) [r:n]
323 LCZero_07ID150  Jellyfish_11-64 (A20) 2018.04.27 =-= (112) [r:n]
551 LCZero_07ID150  Counter_12-64   (D08) 2018.05.01 =-= (115) [r:n]
490 Monolith_04-64  LCZero_07ID150  (A21) 2018.04.30 =-= (123) [b:r]
536 Danasah_70      LCZero_07ID150  (A01) 2018.05.01 =-= (167) [b:r]
158 Glaurung_201-64 LCZero_07ID181  (D00) 2018.04.25 =-= (107) [b1:r]

rwbc avatar May 02 '18 10:05 rwbc