Devlog of Leorik

[j.t.]

That's a nice mate, at first glance the rook sacrifice looks totally nuts, but THEN THE BISHOP COMES and there is nothing more to be done here.

[Mike Sherwin]

I fixed the issue with the TT size and decided that I have everything in place now to release Version 2.0!

Here's the complete feature list:

• Pseudo-legal move generator, bitboards and copy/make
• Negamax search with Alpha-Beta Pruning and Iterative Deepening
• Minimal evaluation based on tapered, tuned PSTs only
• Incrementally updated evaluation and Zobrist hash
• Transposition Table with two buckets and aging
• PVS search (null windows)
• Futility pruning
• Null-Move pruning
• Staged move generation
• MVV-LVA sorted captures
• Killers
• History sorted quiets with LMR

That means Leorik has feature parity with MinimalChass now except for the evaluation where MMC was a little more sophisticated. However Leorik makes that up with raw speed. And not in my wildest dreams did I think that my rewrite would be 7x faster than my first engine. (~7M nps on my computer)

As for what the recent additions mean for Leorik 1.0... this is the match where the engines play without an opening book. I can't help but imagine Leorik 2 to laugh evilly when he's making these two sacrifices in the end.

[pgn][Event "?"]
[Site "?"]
[Date "2022.03.11"]
[Round "?"]
[White "Leorik 1.0"]
[Black "Leorik 2.0"]
[Result "0-1"]
[ECO "C46"]
[GameDuration "00:02:05"]
[GameEndTime "2022-03-11T22:44:09.192 Mitteleuropäische Zeit"]
[GameStartTime "2022-03-11T22:42:03.941 Mitteleuropäische Zeit"]
[Opening "Four knights game"]
[PlyCount "46"]
[TimeControl "40/60+1"]

1. e4 {+0.08/10 5.9s} e5 {-0.09/17 2.7s} 2. Nf3 {+0.18/9 2.7s}
Nc6 {-0.30/16 3.7s} 3. Nc3 {+0.10/8 0.86s} Nf6 {-0.13/17 4.3s}
4. Bd3 {+0.09/8 1.2s} g6 {-0.01/15 2.1s} 5. O-O {+0.16/9 2.2s}
Bg7 {+0.03/16 2.3s} 6. a3 {+0.07/9 3.3s} O-O {+0.21/17 2.7s}
7. Nd5 {+0.02/9 2.2s} d6 {+0.15/17 2.5s} 8. Re1 {-0.02/9 1.9s}
Be6 {+0.20/16 2.0s} 9. c4 {-0.09/9 3.2s} Nd4 {+0.09/16 6.8s}
10. Nxd4 {-0.03/9 1.2s} exd4 {+0.18/15 1.5s} 11. Nf4 {-0.15/8 1.1s}
Bg4 {+0.11/14 2.0s} 12. f3 {-0.08/9 2.0s} Be6 {+0.06/16 5.7s}
13. b4 {-0.02/8 1.3s} Nd7 {+0.26/16 2.2s} 14. Bb2 {-0.13/9 2.3s}
Qg5 {+0.30/16 3.1s} 15. Nxe6 {-0.32/9 4.5s} fxe6 {+0.42/16 2.3s}
16. Kh1 {-0.39/9 5.4s} Ne5 {+0.44/15 2.5s} 17. Bf1 {-0.39/9 7.2s}
Rf4 {+0.33/14 6.5s} 18. d3 {-0.30/8 1.2s} Rh4 {+1.24/15 2.9s}
19. Be2 {-0.67/8 1.6s} Qg3 {+M9/14 1.00s} 20. h3 {-M8/8 1.0s} Ng4 {+M7/16 2.0s}
21. fxg4 {-M6/9 1.3s} Rxh3+ {+M5/17 1.4s} 22. gxh3 {-M4/10 1.4s}
Be5 {+M3/19 2.0s} 23. Bf1 {-M2/11 4.2s} Qh2# {+M1/20 2.0s, Black mates} 0-1[/pgn]

But Leorik 1.0 has it's merits too as it's solving this position in milliseconds that even Stockfish can't solve! (originally found in the expositor thread)

4. Bd3 Is a terrible move, instant advantage for black.
4 ... g6 Generally it is okay to enter into a king's Indian setup but it does not punish white's Bd3. Bc5, Na4, Bb6, Nxb6, Pa7xB and whites knight moved three times and the black bishop only two. And now blacks rook is developed without even moving once.
6 a3 waste a tempo protecting the WB from the BN as the WB should just move to c4.
7 Nd5 White continues to ignore his development and gives black a strong initiative.
7 .. d6 is not a good move. It is thematic in king's Indian positions to move the c6 N to e7 to support f7f5. Nxd5 would expedite that and also it removes the e4 pawn that is needed to restrain the f7 pawn.
8 Re1 does not stop the previous threat and gives the initiative back to black
11 Nc4 another waste of time because taking the bishop opens ups lines to the white king that has no defenders.
11 .. Bg4 also a big waste of time as it allows Be2 for free undoing the damage of Bd3
13 Nd7!
14 Bb2? Bf1!
15 Nxe6? Opening lines against the WK. White is probably lost.
16 Kh1?
17 Bf1?? Finally but now it just loses.
17 .. Rf4!

Leorick2 played some really nice moves!

[eligolf]

I fixed the issue with the TT size and decided that I have everything in place now to release Version 2.0!

Here's the complete feature list:

• Pseudo-legal move generator, bitboards and copy/make
• Negamax search with Alpha-Beta Pruning and Iterative Deepening
• Minimal evaluation based on tapered, tuned PSTs only
• Incrementally updated evaluation and Zobrist hash
• Transposition Table with two buckets and aging
• PVS search (null windows)
• Futility pruning
• Null-Move pruning
• Staged move generation
• MVV-LVA sorted captures
• Killers
• History sorted quiets with LMR

That means Leorik has feature parity with MinimalChass now except for the evaluation where MMC was a little more sophisticated. However Leorik makes that up with raw speed. And not in my wildest dreams did I think that my rewrite would be 7x faster than my first engine. (~7M nps on my computer)

As for what the recent additions mean for Leorik 1.0... this is the match where the engines play without an opening book. I can't help but imagine Leorik 2 to laugh evilly when he's making these two sacrifices in the end.

[pgn][Event "?"]
[Site "?"]
[Date "2022.03.11"]
[Round "?"]
[White "Leorik 1.0"]
[Black "Leorik 2.0"]
[Result "0-1"]
[ECO "C46"]
[GameDuration "00:02:05"]
[GameEndTime "2022-03-11T22:44:09.192 Mitteleuropäische Zeit"]
[GameStartTime "2022-03-11T22:42:03.941 Mitteleuropäische Zeit"]
[Opening "Four knights game"]
[PlyCount "46"]
[TimeControl "40/60+1"]

1. e4 {+0.08/10 5.9s} e5 {-0.09/17 2.7s} 2. Nf3 {+0.18/9 2.7s}
Nc6 {-0.30/16 3.7s} 3. Nc3 {+0.10/8 0.86s} Nf6 {-0.13/17 4.3s}
4. Bd3 {+0.09/8 1.2s} g6 {-0.01/15 2.1s} 5. O-O {+0.16/9 2.2s}
Bg7 {+0.03/16 2.3s} 6. a3 {+0.07/9 3.3s} O-O {+0.21/17 2.7s}
7. Nd5 {+0.02/9 2.2s} d6 {+0.15/17 2.5s} 8. Re1 {-0.02/9 1.9s}
Be6 {+0.20/16 2.0s} 9. c4 {-0.09/9 3.2s} Nd4 {+0.09/16 6.8s}
10. Nxd4 {-0.03/9 1.2s} exd4 {+0.18/15 1.5s} 11. Nf4 {-0.15/8 1.1s}
Bg4 {+0.11/14 2.0s} 12. f3 {-0.08/9 2.0s} Be6 {+0.06/16 5.7s}
13. b4 {-0.02/8 1.3s} Nd7 {+0.26/16 2.2s} 14. Bb2 {-0.13/9 2.3s}
Qg5 {+0.30/16 3.1s} 15. Nxe6 {-0.32/9 4.5s} fxe6 {+0.42/16 2.3s}
16. Kh1 {-0.39/9 5.4s} Ne5 {+0.44/15 2.5s} 17. Bf1 {-0.39/9 7.2s}
Rf4 {+0.33/14 6.5s} 18. d3 {-0.30/8 1.2s} Rh4 {+1.24/15 2.9s}
19. Be2 {-0.67/8 1.6s} Qg3 {+M9/14 1.00s} 20. h3 {-M8/8 1.0s} Ng4 {+M7/16 2.0s}
21. fxg4 {-M6/9 1.3s} Rxh3+ {+M5/17 1.4s} 22. gxh3 {-M4/10 1.4s}
Be5 {+M3/19 2.0s} 23. Bf1 {-M2/11 4.2s} Qh2# {+M1/20 2.0s, Black mates} 0-1[/pgn]

But Leorik 1.0 has it's merits too as it's solving this position in milliseconds that even Stockfish can't solve! (originally found in the expositor thread)

4. Bd3 Is a terrible move, instant advantage for black.
4 ... g6 Generally it is okay to enter into a king's Indian setup but it does not punish white's Bd3. Bc5, Na4, Bb6, Nxb6, Pa7xB and whites knight moved three times and the black bishop only two. And now blacks rook is developed without even moving once.
6 a3 waste a tempo protecting the WB from the BN as the WB should just move to c4.
7 Nd5 White continues to ignore his development and gives black a strong initiative.
7 .. d6 is not a good move. It is thematic in king's Indian positions to move the c6 N to e7 to support f7f5. Nxd5 would expedite that and also it removes the e4 pawn that is needed to restrain the f7 pawn.
8 Re1 does not stop the previous threat and gives the initiative back to black
11 Nc4 another waste of time because taking the bishop opens ups lines to the white king that has no defenders.
11 .. Bg4 also a big waste of time as it allows Be2 for free undoing the damage of Bd3
13 Nd7!
14 Bb2? Bf1!
15 Nxe6? Opening lines against the WK. White is probably lost.
16 Kh1?
17 Bf1?? Finally but now it just loses.
17 .. Rf4!

Leorick2 played some really nice moves!

I think the opening moves are considered questionable at times since he only uses tuned PST as evaluation. I use the same (taken from PeSTO) and mine plays exactly the same. I think it is due to d3 and e3 being slightly higher valued for bishops than c4/f4 which makes it play this way. It has no concept of blocking in own pawns or bad general structures. An opening book would solve much of the issues, or add more evaluation features

[lithander]

Graham included Leorik to play in the currently running Division 7 on very short notice!

So far Leorik isn't exactly playing as successfully as I would have hoped. But the opponents are strong and with a just a few games the loss streak might be due to a statistical fluke. Just bad luck.

But I watched the games played closely and in one game against RookieMonster (Round 2.3) Leorik lost a drawn game by playing an unexplicably bad move 105. ..Kd4

[pgn][Event "92nd Amateur D7"]
[Site "ChessGUI3"]
[Date "2022.03.12"]
[Round "2.3"]
[White "RookieMonster 1.9.6-dev 64-bit"]
[Black "Leorik 2.0 64-bit"]
[Result "1-0"]
[Time "1:44:24 PM"]
[ECO "A40"]
[Opening "Franco-Indian Defense"]
[TimeControl "40/960:40/960:40/960"]
[PlyCount "225"]
[Number "9"]
[Termination "GUI TB Mate in 12"]
[WhiteType "program"]
[BlackType "program"]
[Variant "normal"]

{ i7 Quad }
1.d4 {[%eval 0,1] [%emt 00:00:00]} e6 {[%eval 0,1] [%emt 00:00:00]}
2.e4 {[%eval 0,1] [%emt 00:00:00]} d5 {[%eval 0,1] [%emt 00:00:00]}
3.Nd2 {[%eval 0,1] [%emt 00:00:00]} c5 {[%eval 0,1] [%emt 00:00:00]}
4.exd5 {[%eval 0,1] [%emt 00:00:00]} Qxd5 {[%eval 0,1] [%emt 00:00:00]}
5.Ngf3 {[%eval 0,1] [%emt 00:00:00]} Nf6 {[%eval 0,1] [%emt 00:00:00]}
6.Bc4 {[%eval 0,1] [%emt 00:00:00]} Qd6 {[%eval 0,1] [%emt 00:00:00]}
7.dxc5 {[%eval 0,1] [%emt 00:00:00]} Qxc5 {[%eval 0,1] [%emt 00:00:00]}
8.Qe2 {[%eval 0,1] [%emt 00:00:00]} Bd6 {[%eval 0,1] [%emt 00:00:00]}
9.Nb3 {[%eval 100,19] [%emt 00:00:27]} Qc7 {(Qc7) [%eval -17,16] [%emt 00:00:12]}
10.Bg5 {(Bg5) [%eval 83,18] [%emt 00:00:27]} Nc6 {(O-O) [%eval -8,15] [%emt 00:00:09]}
11.Bxf6 {(O-O-O) [%eval 97,18] [%emt 00:00:27]} gxf6 {(gxf6) [%eval -6,17] [%emt 00:00:13]}
12.O-O-O {(O-O-O) [%eval 80,17] [%emt 00:00:27]} Bd7 {[%eval -14,16] [%emt 00:00:23]}
13.Kb1 {(Rhe1) [%eval 64,19] [%emt 00:00:27]} O-O-O {(O-O-O) [%eval -5,17] [%emt 00:00:18]}
14.Rhe1 {(Nbd2) [%eval 78,18] [%emt 00:00:27]} Ne5 {(Ne7) [%eval -14,16] [%emt 00:00:08]}
15.Nxe5 {(Nxe5) [%eval 50,19] [%emt 00:00:27]} Bxe5 {(fxe5) [%eval -15,16] [%emt 00:00:10]}
16.g3 {(Bd3) [%eval 88,17] [%emt 00:00:27]} Kb8 {(Kb8) [%eval -12,16] [%emt 00:00:13]}
17.c3 {(Nd4) [%eval 79,18] [%emt 00:00:27]} Ba4 {(Ba4) [%eval -10,18] [%emt 00:00:29]}
18.Bb5 {(Bd3) [%eval 76,20] [%emt 00:00:27]} Bxb5 {(Bxb3) [%eval -2,20] [%emt 00:00:19]}
19.Qxb5 {(Qxb5) [%eval 86,21] [%emt 00:00:27]} Qb6 {(a6) [%eval 1,19] [%emt 00:00:13]}
20.Qxb6 {(Qxb6) [%eval 147,22] [%emt 00:00:27]} axb6 {(axb6) [%eval 0,20] [%emt 00:00:15]}
21.Rxd8 {(Kc2) [%eval 137,23] [%emt 00:00:27]} Rxd8 {(Rxd8) [%eval -8,22] [%emt 00:00:21]}
22.Kc2 {(Kc2) [%eval 143,23] [%emt 00:00:27]} Bc7 {(Kc7) [%eval -9,22] [%emt 00:01:13]}
23.Re4 {(Nd4) [%eval 144,23] [%emt 00:00:27]} Rd5 {(f5) [%eval -9,21] [%emt 00:00:26]}
24.Rg4 {(Nd4) [%eval 188,23] [%emt 00:00:27]} Rf5 {(Rg5) [%eval -9,20] [%emt 00:00:22]}
25.Rg8 {(Rg8) [%eval 214,24] [%emt 00:00:27]} Ka7 {(Ka7) [%eval -29,23] [%emt 00:00:14]}
26.Nd4 {(f4) [%eval 199,26] [%emt 00:00:27]} Rxf2 {(Rxf2) [%eval -38,24] [%emt 00:00:23]}
27.Kb3 {(Kb3) [%eval 186,26] [%emt 00:00:27]} Bb8 {(b5) [%eval -60,26] [%emt 00:00:20]}
28.Nb5 {(Nb5) [%eval 193,26] [%emt 00:00:27]} Ka8 {(Ka8) [%eval -57,27] [%emt 00:00:14]}
29.Rg4 {(Rg4) [%eval 239,28] [%emt 00:00:27]} Be5 {(Be5) [%eval -45,26] [%emt 00:00:13]}
30.Rh4 {(Rh4) [%eval 232,27] [%emt 00:00:27]} h6 {(Kb8) [%eval -35,27] [%emt 00:00:33]}
31.Rxh6 {(Rxh6) [%eval 242,26] [%emt 00:00:27]} f5 {(f5) [%eval -57,26] [%emt 00:00:28]}
32.Rh7 {(Rh7) [%eval 266,26] [%emt 00:00:27]} f4 {(f4) [%eval -50,26] [%emt 00:00:44]}
33.g4 {(Rh5) [%eval 266,24] [%emt 00:00:27]} f6 {[%eval -40,25] [%emt 00:00:51]}
34.Rh8 {(h3) [%eval 208,23] [%emt 00:00:27]} Bb8 {(Bb8) [%eval -25,25] [%emt 00:00:31]}
35.Nc7 {(Rh6) [%eval 232,24] [%emt 00:00:27]} Ka7 {(Ka7) [%eval 0,27] [%emt 00:00:40]}
36.Nxe6 {(Nb5) [%eval 233,24] [%emt 00:00:38]} f3 {(f3) [%eval 0,26] [%emt 00:00:59]}
37.Rh3 {(Nd4) [%eval 227,23] [%emt 00:00:25]} Bxh2 {(Bxh2) [%eval -91,24] [%emt 00:00:30]}
38.g5 {(g5) [%eval 188,23] [%emt 00:00:25]} fxg5 {(fxg5) [%eval -88,25] [%emt 00:00:47]}
39.Nxg5 {(Nxg5) [%eval 192,27] [%emt 00:00:25]} Kb8 {(Bf4) [%eval -92,25] [%emt 00:00:33]}
40.Nxf3 {(Rxf3) [%eval 295,22] [%emt 00:00:25]} Bf4 {(Bf4) [%eval -99,26] [%emt 00:02:08]}
41.Rh8 {(Ne1) [%eval 292,23] [%emt 00:00:24]} Kc7 {(Kc7) [%eval -90,25] [%emt 00:00:15]}
42.Rh7 {(Nd4) [%eval 272,23] [%emt 00:00:24]} Kc8 {(Kb8) [%eval -95,25] [%emt 00:00:14]}
43.Nd4 {(Nd4) [%eval 298,22] [%emt 00:00:24]} Bc1 {(Bc1) [%eval -102,24] [%emt 00:00:45]}
44.Nc2 {(Nc2) [%eval 298,20] [%emt 00:00:24]} Bg5 {[%eval -108,23] [%emt 00:00:33]}
45.Rh8 {(a4) [%eval 305,21] [%emt 00:00:24]} Kd7 {(Kc7) [%eval -108,24] [%emt 00:00:23]}
46.Rh5 {(a4) [%eval 302,23] [%emt 00:00:24]} Bc1 {(Bc1) [%eval -108,24] [%emt 00:00:14]}
47.Rh7 {(Rh1) [%eval 298,22] [%emt 00:00:24]} Kc8 {(Kc8) [%eval -111,24] [%emt 00:00:21]}
48.Rh8 {(a4) [%eval 298,23] [%emt 00:00:24]} Kd7 {(Kc7) [%eval -108,25] [%emt 00:00:37]}
49.a3 {(Rh1) [%eval 300,22] [%emt 00:00:24]} Re2 {(Bg5) [%eval -108,23] [%emt 00:00:15]}
50.Rf8 {(a4) [%eval 305,20] [%emt 00:00:24]} Re8 {(Kc7) [%eval -108,23] [%emt 00:00:11]}
51.Rf1 {(Rf5) [%eval 311,20] [%emt 00:00:24]} Bg5 {(Bg5) [%eval -111,22] [%emt 00:00:11]}
52.Rf5 {(Rf7) [%eval 311,19] [%emt 00:00:24]} Be7 {(Bh4) [%eval -116,24] [%emt 00:01:05]}
53.Nd4 {(Kc4) [%eval 318,21] [%emt 00:00:24]} Rf8 {(Bd8) [%eval -111,25] [%emt 00:00:18]}
54.Rd5 {(Rxf8) [%eval 312,20] [%emt 00:00:24]} Kc8 {(Kc8) [%eval -118,24] [%emt 00:00:28]}
55.Rb5 {(Kc4) [%eval 301,20] [%emt 00:00:24]} Bd8 {(Rf6) [%eval -117,24] [%emt 00:00:18]}
56.Ne6 {(Kc4) [%eval 333,26] [%emt 00:00:24]} Rf6 {(Rf6) [%eval -98,25] [%emt 00:00:14]}
57.Nxd8 {(Nxd8) [%eval 336,27] [%emt 00:00:24]} Kxd8 {(Kxd8) [%eval -116,28] [%emt 00:00:12]}
58.Rd5 {(Kc4) [%eval 328,27] [%emt 00:00:24]} Kc7 {(Kc7) [%eval -114,29] [%emt 00:00:14]}
59.Kb4 {(Kc4) [%eval 328,30] [%emt 00:00:24]} Kc6 {(Kb8) [%eval -112,31] [%emt 00:00:20]}
60.Rg5 {(Rd4) [%eval 328,25] [%emt 00:00:24]} Rf4 {(Kc7) [%eval -107,31] [%emt 00:00:23]}
61.c4 {(c4) [%eval 328,28] [%emt 00:00:24]} Rf2 {(Rf6) [%eval -107,27] [%emt 00:00:08]}
62.Rg6 {(Rg6) [%eval 328,31] [%emt 00:00:24]} Kc7 {(Kc7) [%eval -106,29] [%emt 00:00:13]}
63.b3 {(Kb3) [%eval 328,33] [%emt 00:00:24]} Rf5 {(Rf7) [%eval -106,28] [%emt 00:00:07]}
64.a4 {(Rg3) [%eval 328,32] [%emt 00:00:24]} Rf3 {(Rf7) [%eval -103,28] [%emt 00:00:10]}
65.Rg7 {(Rg1) [%eval 328,32] [%emt 00:00:24]} Kc6 {(Kb8) [%eval -105,32] [%emt 00:00:15]}
66.Rg4 {(Rg1) [%eval 328,32] [%emt 00:00:24]} Re3 {(Kc7) [%eval -105,33] [%emt 00:00:17]}
67.Rg6 {(Rg6) [%eval 328,35] [%emt 00:00:24]} Kc7 {(Kc7) [%eval -105,32] [%emt 00:00:09]}
68.Rf6 {(Rg1) [%eval 328,37] [%emt 00:00:24]} Rg3 {(Rg3) [%eval -105,32] [%emt 00:00:15]}
69.Rh6 {(Rf1) [%eval 328,37] [%emt 00:00:24]} Rf3 {(Rf3) [%eval -106,33] [%emt 00:00:55]}
70.Rh5 {(Rh1) [%eval 328,34] [%emt 00:00:24]} Kc6 {(Kd6) [%eval -106,30] [%emt 00:00:10]}
71.Ka3 {(Rh2) [%eval 328,29] [%emt 00:00:24]} Kd6 {(Kc7) [%eval -106,30] [%emt 00:00:18]}
72.Rd5 {(Rd5) [%eval 328,31] [%emt 00:00:23]} Kc6 {(Kc7) [%eval -106,30] [%emt 00:00:18]}
73.Rg5 {(Kb2) [%eval 328,29] [%emt 00:00:24]} Kd6 {(Kc7) [%eval -106,31] [%emt 00:00:26]}
74.Kb4 {(Kb4) [%eval 328,36] [%emt 00:00:24]} Re3 {(Kc7) [%eval -106,32] [%emt 00:00:22]}
75.Rf5 {(Rg2) [%eval 328,34] [%emt 00:00:23]} Kc6 {(Kc7) [%eval -106,33] [%emt 00:00:31]}
76.Rf6 {(Rf1) [%eval 328,35] [%emt 00:00:23]} Kc7 {(Kc7) [%eval -106,32] [%emt 00:00:17]}
77.Rf7 {(Rf2) [%eval 328,36] [%emt 00:00:23]} Kc6 {(Kb8) [%eval -106,34] [%emt 00:00:26]}
78.Rf5 {(Rf1) [%eval 328,34] [%emt 00:00:23]} Rg3 {(Kc7) [%eval -105,33] [%emt 00:00:32]}
79.Re5 {(Rf2) [%eval 328,34] [%emt 00:00:23]} Kd6 {(Rf3) [%eval -106,33] [%emt 00:00:39]}
80.Rh5 {(Re1) [%eval 328,36] [%emt 00:00:23]} Kc6 {(Rf3) [%eval -106,35] [%emt 00:01:44]}
81.Rh6 {(Rh2) [%eval 328,36] [%emt 00:00:24]} Kc7 {(Kc7) [%eval -106,33] [%emt 00:00:16]}
82.Rh7 {(Rh2) [%eval 328,35] [%emt 00:00:24]} Kc6 {[%eval -106,34] [%emt 00:00:20]}
83.Re7 {(Re7) [%eval 328,35] [%emt 00:00:24]} Rh3 {(Rf3) [%eval -106,32] [%emt 00:00:13]}
84.Re6 {(Re1) [%eval 328,36] [%emt 00:00:24]} Kc7 {(Kc7) [%eval -106,31] [%emt 00:00:12]}
85.Re8 {(Re5) [%eval 328,36] [%emt 00:00:24]} Kd6 {(Rf3) [%eval -106,33] [%emt 00:00:24]}
86.Rd8 {(Re1) [%eval 328,30] [%emt 00:00:24]} Kc6 {(Kc7) [%eval -106,33] [%emt 00:00:16]}
87.Ra8 {(Ra8) [%eval 328,32] [%emt 00:00:23]} Rh1 {(Kc7) [%eval -106,32] [%emt 00:00:20]}
88.Rc8 {(Rc8) [%eval 328,32] [%emt 00:00:23]} Kd7 {(Kd6) [%eval -106,31] [%emt 00:00:10]}
89.Rg8 {(Rf8) [%eval 328,34] [%emt 00:00:24]} Rh5 {(Kc6) [%eval -106,31] [%emt 00:00:13]}
90.Rg7 {(Rg2) [%eval 328,35] [%emt 00:00:24]} Kc6 {(Kc6) [%eval -106,30] [%emt 00:00:09]}
91.Rf7 {(Rg3) [%eval 328,35] [%emt 00:00:24]} Rg5 {(Rh3) [%eval -106,31] [%emt 00:00:09]}
92.Re7 {(Re7) [%eval 328,30] [%emt 00:00:24]} Rg3 {(Rg3) [%eval -106,33] [%emt 00:00:28]}
93.Rh7 {(Re6) [%eval 328,30] [%emt 00:00:24]} Re3 {(Rg2) [%eval -106,30] [%emt 00:00:22]}
94.Rf7 {(Rh2) [%eval 328,31] [%emt 00:00:23]} Rg3 {(Re5) [%eval -106,32] [%emt 00:00:17]}
95.Ka3 {(Rf8) [%eval 328,28] [%emt 00:00:24]} Rg1 {(Re3) [%eval -106,29] [%emt 00:00:16]}
96.Rh7 {(Rf6) [%eval 328,26] [%emt 00:00:24]} Rg3 {(Ra1) [%eval -106,30] [%emt 00:00:12]}
97.Rh6 {(Rh8) [%eval 328,27] [%emt 00:00:24]} Kc5 {(Kc7) [%eval -106,33] [%emt 00:00:41]}
98.Rh5 {(Rh5) [%eval 328,28] [%emt 00:00:24]} Kc6 {(Kd6) [%eval -105,32] [%emt 00:00:58]}
99.Rf5 {(Rh8) [%eval 328,28] [%emt 00:00:24]} Rh3 {(Kd6) [%eval -105,29] [%emt 00:00:30]}
100.Rf1 {(Rf8) [%eval 328,28] [%emt 00:00:24]} Kc5 {(Re3) [%eval -105,30] [%emt 00:00:16]}
101.Rf6 {(Rf7) [%eval 328,27] [%emt 00:00:24]} Rh5 {(Rg3) [%eval -105,27] [%emt 00:00:26]}
102.Rf7 {(Rf8) [%eval 328,27] [%emt 00:00:23]} Kc6 {(Kc6) [%eval -9,27] [%emt 00:00:13]}
103.Re7 {(Rf3) [%eval 328,29] [%emt 00:00:24]} Rh6 {(Rg5) [%eval -4,26] [%emt 00:00:12]}
104.Re3 {(c5) [%eval 262,29] [%emt 00:00:50]} Kc5 {(Kc7) [%eval 0,33] [%emt 00:00:21]}
105.Rf3 {(Re8) [%eval 262,28] [%emt 00:00:22]} Kd4 {(Kc6) [%eval 0,30] [%emt 00:00:13]}
106.Kb4 {(Kb4) [%eval 560,21] [%emt 00:00:22]} Rc6 {(Rh7) [%eval 0,40] [%emt 00:00:21]}
107.Rf7 {(Rf7) [%eval 743,19] [%emt 00:00:22]} Rc5 {(Rd6) [%eval 0,39] [%emt 00:00:10]}
108.Rd7 {(Rd7) [%eval 998,21] [%emt 00:00:22]} Ke5 {[%eval 0,99] [%emt 00:00:02]}
109.Rd5 {(Rxb7) [%eval 3020,19] [%emt 00:00:22]} Ke4 {[%eval 0,48] [%emt 00:00:25]}
110.Rxc5 {[%eval 99960,20] [%emt 00:00:24]} bxc5 {(Kd3) [%eval -#17,40] [%emt 00:00:41]}
111.Kxc5 {(Kxc5) [%eval 99970,7] [%emt 00:00:00]} Kd3 {(Kf5) [%eval -#15,39] [%emt 00:00:18]}
112.Kb6 {(a5) [%eval 99972,5] [%emt 00:00:00]} Kc3 {(Kc2) [%eval -#14,40] [%emt 00:00:50]}
113.Kxb7 {(Kxb7) [%eval 99974,2] [%emt 00:00:00]}
1-0[/pgn]

When I feed the position directly to the engine it never even considers to move the king but has only reasonable rook moves in the PV

Code: Select all

position fen 8/1p6/1p5r/2k5/P1P5/KP3R2/8/8 b - - 82 105
go
info depth 1 score cp -34 nodes 17 nps 1700 time 10 pv h6h1
info depth 2 score cp -55 nodes 72 nps 6545 time 11 pv h6h1 f3f8
info depth 3 score cp -59 nodes 636 nps 57818 time 11 pv h6h1 f3f7 h1a1
info depth 4 score cp -59 nodes 761 nps 63416 time 12 pv h6h1 f3f7 h1a1 a3b2
info depth 5 score cp -65 nodes 1823 nps 151916 time 12 pv h6h7 f3f5 c5d4 f5b5 h7h6
info depth 6 score cp -73 nodes 3822 nps 318500 time 12 pv h6h2 f3f7 c5c6 a3b4 h2c2 f7f8
info depth 7 score cp -68 nodes 5140 nps 395384 time 13 pv h6h2 f3f7 c5c6 a3b4 h2c2 f7g7 c2c1
info depth 8 score cp -83 nodes 10840 nps 774285 time 14 pv h6h1 f3f7 c5c6 a3b4 h1c1 f7f6 c6c7 b4b5
info depth 9 score cp -80 nodes 19406 nps 1141529 time 17 pv h6d6 f3f7 c5c6 a3b4 d6d3 f7g7 d3d7 g7g6 d7d6
info depth 10 score cp -83 nodes 28545 nps 1502368 time 19 pv h6h1 f3f7 c5c6 a3b4 h1e1 f7f6 c6c7 b4b5 e1e5 b5b4
info depth 11 score cp -80 nodes 69524 nps 2242709 time 31 pv h6d6 f3f7 c5c6 a3b4 d6d3 f7f6 c6c7 f6f8 c7d7 f8b8 d7c6
info depth 12 score cp -92 nodes 197877 nps 2953388 time 67 pv h6h1 f3f7 c5c6 b3b4 h1a1 a3b3 a1b1 b3c3 b1c1 c3d3 c1d1 d3e4
info depth 13 score cp -92 nodes 230398 nps 2992181 time 77 pv h6h1 f3f7 c5c6 b3b4 h1a1 a3b3 a1b1 b3c3 b1c1 c3d3 c1d1 d3e4 d1c1
info depth 14 score cp -91 nodes 519675 nps 3352741 time 155 pv h6h7 f3f5 c5d6 b3b4 h7h3 a3b2 d6e6 f5b5 h3h2 b2c3 h2h3 c3d4 h3h4 d4e3
info depth 15 score cp -90 nodes 638071 nps 3376037 time 189 pv h6h7 f3f5 c5d6 b3b4 h7h3 a3b2 h3h2 b2c3 h2h3 c3d4 h3h4 d4d3 h4h3 d3e4 h3h2
info depth 16 score cp -96 nodes 830676 nps 3446788 time 241 pv h6h7 f3f5 c5d6 f5d5 d6c6 b3b4 h7h3 a3b2 h3h2 b2c3 h2a2 a4a5 b6a5 d5a5 a2g2 c4c5
info depth 17 score cp -91 nodes 1045444 nps 3484813 time 300 pv h6h7 f3f5 c5d6 f5d5 d6c6 b3b4 h7h3 a3b2 h3h2 b2c3 h2a2 a4a5 b6a5 b4a5 b7b6 a5b6 c6b6
info depth 18 score cp -102 nodes 1487788 nps 3834505 time 388 pv h6h7 f3f5 c5d6 f5d5 d6c6 b3b4 h7h3 a3b2 h3h2 b2c3 h2a2 a4a5 b6a5 b4a5 a2a1 c3d4 a1d1 d4e4
info depth 19 score cp -89 nodes 2258205 nps 4427852 time 510 pv h6h7 f3f5 c5d6 f5d5 d6c6 b3b4 h7h3 a3b2 h3h2 b2c3 h2h3 d5d3 h3h1 a4a5 b6a5 b4a5 h1c1 c3d4 c6d6
info depth 20 score cp -85 nodes 5281050 nps 5102463 time 1035 pv h6h7 f3f5 c5d6 f5d5 d6c6 d5b5 h7h2 a3b4 h2e2 b5g5 e2e3 g5f5 e3h3 f5e5 h3d3 e5e8 c6d7 e8b8 d7c6 b8e8
info depth 21 score cp -82 nodes 8699071 nps 5301079 time 1641 pv h6h7 f3f5 c5d6 f5d5 d6c6 b3b4 h7h3 a3b2 h3h2 b2c3 h2h3 d5d3 h3h1 b4b5 c6c7 d3d5 h1a1 c3b4 a1b1 b4a3 b1c1
info depth 22 score cp -82 nodes 10544740 nps 5288234 time 1994 pv h6h7 f3f5 c5d6 f5d5 d6c6 b3b4 h7h3 a3b2 h3h2 b2c3 h2h3 d5d3 h3h1 b4b5 c6c7 d3d5 h1a1 c3b4 a1b1 b4a3 b1c1 a3b4
info depth 23 score cp -83 nodes 14008405 nps 5387848 time 2600 pv h6h7 f3f5 c5d6 f5d5 d6c6 a3b4 h7h3 a4a5 b6a5 d5a5 b7b6 a5d5 h3h1 b4c3 h1c1 c3d4 c1b1 d5b5 b1e1 b5d5 e1d1 d4e4 d1c1
info depth 24 score cp -82 nodes 56307262 nps 5401176 time 10425 pv h6h1 f3f7 c5c6 f7g7 h1h5 g7e7 h5h3 e7e6 c6c7 e6e5 h3d3 e5g5 c7d6 g5b5 d6c6 a3b4 d3e3 a4a5 b6a5 b4a5 e3e1 b5g5 b7b6 a5b4
info depth 25 score cp -82 nodes 69339581 nps 5374744 time 12901 pv h6h1 f3f7 c5c6 f7g7 h1h5 g7e7 h5h3 e7e6 c6c7 e6e5 c7c6 a3b4 h3g3 a4a5 b6a5
info depth 26 score cp -85 nodes 133443105 nps 5295991 time 25197 pv h6h1 f3f7 c5c6 f7f6 c6c7 f6f5 c7c6 a3b4 h1h3 f5f6 c6c7 f6f5 c7c6 f5f6 c6c7 f6f5 c7c6 f5f6 c6c7 f6f5 c7c6 f5f6 c6c7 f6f5 c7c6 f5f6
info depth 27 score cp -86 nodes 181754178 nps 5293245 time 34337 pv h6h1 f3f5 c5c6 f5b5 h1h3 a3b4 h3g3 b5d5 g3h3 a4a5 b6a5 d5a5 b7b6 a5g5 h3f3 g5g8 c6b7 g8g7 b7c6 g7e7 f3f4 e7e8 f4f3

I didn't find an explanation other than that the transposition table must have stored wrong data. I found an issue where an aborted search (a decision from the time control code) would still write the results of this incomplete search to the TT which potentially could be the cause. I couldn't reproduce the error but still replaced the binaries on github with a fixed version. In my own fast time control tests I did overnight it makes only small difference but on longer time controls it might be more impactful.

Code: Select all

Score of Leorik 2.0.2 vs Leorik 2.0: 1109 - 1014 - 1669  [0.513] 3792
...      Leorik 2.0.2 playing White: 605 - 442 - 849  [0.543] 1896
...      Leorik 2.0.2 playing Black: 504 - 572 - 820  [0.482] 1896
...      White vs Black: 1177 - 946 - 1669  [0.530] 3792
Elo difference: 8.7 +/- 8.3, LOS: 98.0 %, DrawRatio: 44.0 %

So I stealthy updated the binary releases with the new version which should read Leorik-2.0.2

[lithander]

Graham included Leorik to play in the currently running Division 7 on very short notice!

I had asked Graham to replace MinimalChess playing in Division 8 with Leorik but also cautioned him that Leorik would be at elast 100 Elo stronger. So he put Leorik into Division 7 where I would have thought it would find equal opponents but it turns out it can't really keep up. MinimalChess couldn't do any worse there.... so how could I misjudge the engines strength by such a large margin?

Well, I was playing my own matches on much faster time controls. And it seems that Leorik overperforms on fast time controls and underperforms on slow time controls. Lars who's often around in the chats of the matches Graham broadcasts has put that theory to a test. He's access to a lot of compute power and maintains his own rating list and he put Leorik up in a gauntlet of 90 opponents on fast 1 minute games playing almost four thousand games.

Code: Select all

[Lars] - 163 Leorik 2.0.2 : 2637 10 10 3798 51.6 % 2626 16.6 %
[Lars] - 221 Minimal 0.6 : 2507 11 11 3248 46.0 % 2535 19.2 %
[Lars] - plus 130 elo at 1 minute

So this confirms my own measurements pretty accurately.

Then he repeated the test with more than one thousand 5 minute games.

Code: Select all

[Lars] - 155 Leorik 2.0.2 : 2607 18 18 1198 44.4 % 2646 18.7 %
[Lars] - 185 Minimal 0.6 : 2527 18 18 1091 49.5 % 2530 23.8 %
[Lars] - only plus 80 elo at 5 minutes

So that's 50 Elo less due to giving 5x more time to the engines. And it appears that this trend continues the more time the engines have available.
Currently Graham is also running a Gauntlet with Leorik in the CCRL 40/15 list that you can watch here and I'm curious to see if the theory holds.

But if it does that's not really surprising in hindsight. If your engine can look 2 plys deeper than its opponent due to speed differences it makes a big difference if that's 10+2 or 20+2. Watching the games I often noticed Leorik and his opponent to disagree on the value of positions very early in the game by at least one or two pawns. Leorik will think he's in an equal position while the opponent is playing moves that are compromising Leoriks pawn structure or King safety or mobility. Something Leorik can't detect due to it's PST only evaluation. Then all of a sudden Leorik will report something bad happening on the horizon and it's evaluation drops often below that of the opponent. But even if Leorik can see the material rewards of the opponent's positional play coming to fruition earlier at that point it's too late to do anything about it.

I'm very happy and grateful for others picking up my engine and giving it such extended opportunity to play! It's a tremendous boost to my motivation and I'm already looking forward to working towards the next major release which should be all about improving the evaluation!

[j.t.]

I noticed something like this as well. My theory is that mainly engines that get a lot of Elo because of NPS speed suffer from this phenomenon, while slow engines with very good search algorithms would get better at long time controls. Simply because to keep up with a better search, the NPS performance would need to be improved superlinearly with more available time (which can't be the case).

[connor_mcmonigle]

At longer time controls, I do believe there is a phenomenon where having an accurate, if slow, evaluation function yields better performance as compared to a fast inaccurate evaluation function even if both yield similar results at shorter TCs. Some Elo compression is, of course, expected at longer time controls so apparent gains will be deflated regardless. Furthermore, I think it's also important to note that poorly tuned time management can result in TC scaling issues. Do MinimalChess and Leorik share the same TM logic? That would be the first place I'd look.

[amanjpro]

I remember I used to have an issue with my history scores exceeding the bounds (due to int16-overflow bug), which only appeard (or badly appeard) in long time controls. You may have something like this too

[lithander]

I noticed something like this as well. My theory is that mainly engines that get a lot of Elo because of NPS speed suffer from this phenomenon, while slow engines with very good search algorithms would get better at long time controls. Simply because to keep up with a better search, the NPS performance would need to be improved superlinearly with more available time (which can't be the case).

This explanation makes a lot of intuitive sense to me. MinimalChess is much slower but it does have a better eval. As the time controls get longer the score gap between Leorik (better) and MinimalChess (worse) should be expected to decrease. So according to that theory the above described results wouldn't be reason for concern.

Furthermore, I think it's also important to note that poorly tuned time management can result in TC scaling issues. Do MinimalChess and Leorik share the same TM logic? That would be the first place I'd look.

I have to admit that in my own tests I usually had an increment and I have now watched the allocation of time in games where the budget has to be spread out over multiple moves. Without an increment both Leorik and MinimalChess are very conservative. They calculate a per-move budget and don't like exceeding it. Because I don't want to abort all searches exactly when the budget runs out I rather not start another iteration if I expect it to exceed the budget. That means that they rarely use up the complete per-move budget.

So if you give them 20 minutes per 40 moves they will only spent an average of 20s on the early moves instead of the 30s which would be the correct per-move budget. As they approach the 40 move barrier the budget per move increases from all this saved up time so they end up spending an average of 40s on these later moves until in move 41 they are back to an average of 20s.
I don't know what would be ideal allocation, either balanced or maybe even a bit front-loaded so you can edge out an advantage early and then you try to keep it despite spending less time on moves later. But I suppose that the status-quo is really not optimal.

I remember I used to have an issue with my history scores exceeding the bounds (due to int16-overflow bug), which only appeard (or badly appeard) in long time controls. You may have something like this too

I have no guarding against overflow in my code so this seemed very likely. But I checked it and based on my tests you would have to search for several hours before the values overflow. While I increment the history scores with depth*depth the most increments happen near the leafs where the remaining depth is rather small. Also whenever I start a new search iteration I scale the history values down by exactly halving them. Maybe not ideally implemented but shouldn't break down on longer games.

[amanjpro]

I noticed something like this as well. My theory is that mainly engines that get a lot of Elo because of NPS speed suffer from this phenomenon, while slow engines with very good search algorithms would get better at long time controls. Simply because to keep up with a better search, the NPS performance would need to be improved superlinearly with more available time (which can't be the case).

This explanation makes a lot of intuitive sense to me. MinimalChess is much slower but it does have a better eval. As the time controls get longer the score gap between Leorik (better) and MinimalChess (worse) should be expected to decrease. So according to that theory the above described results wouldn't be reason for concern.

Furthermore, I think it's also important to note that poorly tuned time management can result in TC scaling issues. Do MinimalChess and Leorik share the same TM logic? That would be the first place I'd look.

I have to admit that in my own tests I usually had an increment and I have now watched the allocation of time in games where the budget has to be spread out over multiple moves. Without an increment both Leorik and MinimalChess are very conservative. They calculate a per-move budget and don't like exceeding it. Because I don't want to abort all searches exactly when the budget runs out I rather not start another iteration if I expect it to exceed the budget. That means that they rarely use up the complete per-move budget.

So if you give them 20 minutes per 40 moves they will only spent an average of 20s on the early moves instead of the 30s which would be the correct per-move budget. As they approach the 40 move barrier the budget per move increases from all this saved up time so they end up spending an average of 40s on these later moves until in move 41 they are back to an average of 20s.
I don't know what would be ideal allocation, either balanced or maybe even a bit front-loaded so you can edge out an advantage early and then you try to keep it despite spending less time on moves later. But I suppose that the status-quo is really not optimal.

I remember I used to have an issue with my history scores exceeding the bounds (due to int16-overflow bug), which only appeard (or badly appeard) in long time controls. You may have something like this too

I have no guarding against overflow in my code so this seemed very likely. But I checked it and based on my tests you would have to search for several hours before the values overflow. While I increment the history scores with depth*depth the most increments happen near the leafs where the remaining depth is rather small. Also whenever I start a new search iteration I scale the history values down by exactly halving them. Maybe not ideally implemented but shouldn't break down on longer games.

I am using int8 for depth, which fills up rather fast. But with int16 you would be safe. So in history score I first convert to int32 then do the math.

Btw I don't change/reset/halve history scores even between games, this was some +10 or so elo points