About a rebel evaluation study

[xr_a_y]

ok, I am running a tourney with stockfish with just psqt. Indeed, stockfish is looking at depth 17 when Weini reaches depth 8 ... this may explain at least 500 elo !

I'll try fixed depth stuff also ...

[hgm]

I don't know. My engine searching 12 ply regularly beats other engines searching 18 ply. We did not invent the term 'hollow plies' for nothing.

[Ras]

I don't know. My engine searching 12 ply regularly beats other engines searching 18 ply. We did not invent the term 'hollow plies' for nothing.

Ah that's easy - prune only the unimportant moves, not the important ones.

[xr_a_y]

Ok ... so sotckfish with just psqt is already just incredible ...

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 stockfish(only psqt)          893     nan      86   99.4%    1.2%
   2 fairymax                      -82      65      86   38.4%   25.6%
   3 weini1thread                 -122      65      86   33.1%   29.1%

[tomitank]

Ok ... so sotckfish with just psqt is already just incredible ...

..and don't forget the search algorithm

[Uri Blass]

Ok ... so sotckfish with just psqt is already just incredible ...

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 stockfish(only psqt)          893     nan      86   99.4%    1.2%
   2 fairymax                      -82      65      86   38.4%   25.6%
   3 weini1thread                 -122      65      86   33.1%   29.1%

Note that it is possible to achieve more than stockfish(only psqt) because stockfish parameters are not optimized for psqt evaluation and I guess the psqt is not the best and also the search is not tuned for this specific evaluation and inspite of all these disadvantages I will not be surprised if stockfish (only piece square table) can get more than 2600 CCRL rating.

[xr_a_y]

In this position

[d]r4kr1/1bq1bp1p/p3p3/5p2/8/2NQN3/PP3PPP/2R2RK1 w - - 2 20

extracted from this game (move 19)

[pgn]
[Event "?"]
[Site "?"]
[Date "2018.05.01"]
[Round "19"]
[White "weini1"]
[Black "stockfish(only psqt)"]
[Result "0-1"]
[ECO "A46"]
[Opening "Queen's pawn game"]
[PlyCount "69"]
[Termination "adjudication"]
[TimeControl "40/20"]

1. d4 {book} Nf6 {book} 2. Nf3 {book} d5 {book} 3. c4 {book} dxc4 {book}
4. e3 {book} c5 {+0.02/19 0.98s} 5. Bxc4 {+0.33/9 0.62s} e6 {0.00/20 0.54s}
6. O-O {+0.44/8 0.62s} Nc6 {0.00/18 0.19s} 7. Bb5 {+0.32/8 0.62s}
a6 {+0.13/18 0.56s} 8. Bxc6+ {+0.46/8 0.62s} bxc6 {-0.10/17 0.35s}
9. Qc2 {+0.37/8 0.61s} cxd4 {+0.24/17 0.21s} 10. exd4 {+0.27/7 0.61s}
Bb7 {+0.13/17 0.30s} 11. Nc3 {+0.23/8 0.61s} c5 {+0.30/16 0.35s}
12. dxc5 {+0.21/8 0.60s} Bxc5 {+0.44/17 0.27s} 13. Bg5 {+0.07/8 0.60s}
Qa5 {+0.48/17 0.70s} 14. Bxf6 {+0.04/9 0.60s} gxf6 {+0.54/14 0.12s}
15. Nd2 {+0.01/8 0.59s} Be7 {+0.52/16 0.85s} 16. Nc4 {+0.03/6 0.59s}
Qc7 {+0.54/17 0.51s} 17. Ne3 {-0.53/7 0.58s} Kf8 {+0.44/17 1.2s}
18. Qd3 {-0.05/6 0.58s} f5 {+0.65/15 0.52s} 19. Rac1 {+0.01/6 0.14s}
Rg8 {+0.70/15 0.28s} 20. g3 {-0.38/6 0.44s} Qc6 {+1.46/17 0.19s}
21. f3 {-0.68/7 0.61s} Rd8 {+1.39/19 0.34s} 22. Qe2 {-0.67/9 0.60s}
f4 {+1.47/20 0.40s} 23. Ncd5 {-1.36/11 0.60s} Qd6 {+1.42/19 0.30s}
24. Nxe7 {-1.36/11 0.59s} Qxe7 {+1.50/19 1.6s} 25. Nc4 {-1.36/9 0.58s}
fxg3 {+1.49/19 0.29s} 26. h3 {-1.22/8 0.30s} g2 {+1.45/21 0.36s}
27. Rf2 {-0.88/7 0.41s} Qh4 {+2.14/18 0.23s} 28. Qe1 {-1.36/9 0.61s}
Qxh3 {+M47/18 0.32s} 29. Qb4+ {-9.79/15 4.2s} Ke8 {+M33/25 0.26s}
30. Qa4+ {-9.79/13 0.24s} Rd7 {+M31/27 0.28s} 31. Nd6+ {-9.79/11 0.24s}
Kf8 {+M13/31 0.23s} 32. Qh4 {-19.35/17 1.7s} Qxh4 {+M11/37 0.30s}
33. Rxg2 {-19.73/15 0.039s} Rxd6 {+M9/43 0.35s} 34. Rxg8+ {-14.91/7 0.015s}
Kxg8 {+M7/58 0.35s} 35. Rc3 {-14.91/5 0.037s, Black wins by adjudication} 0-1
[/pgn]

The only defending move for white seems to be Ne2 but moves like g3, f4 or Na4 are not that bad at first but fail to deep problems. Weini finds this only good move at depth 11 in more than 24sec for 15968079 nodes if LMR is ON (and depth 6 if LMR is off in 2 second looking a more than 1 million of nodes) after that the search is stable. In this short 20sec per TC game, there is no way Weini can find this move ...

Code: Select all

LMR ON
1 -868 0 10 d3a6 
1 -210 0 51 e3f5 
1 -84 0 73 c3d5 
1 -74 0 146 c3e4 
1 -49 0 171 f2f4 
1 -49 0 253 f2f4 
2 6 0 539 f2f4 f8g7 
2 22 0 1188 c3d5 c7d8 
2 22 0 2394 c3d5 c7d8 
3 -214 0 4697 c3d5 c7d6 e3f5 
3 -81 0 7011 f2f4 b7g2 e3g2 
3 -54 1 11268 h2h3 e7c5 d3c4 
3 -54 2 16319 h2h3 e7c5 d3c4 
4 1 4 26635 h2h3 e7d6 g2g3 f8g7 
4 7 5 35386 f1d1 f8g7 c3d5 c7d6 
4 7 6 37623 f1d1 f8g7 c3d5 c7d6 
5 -65 13 73299 f1d1 a8d8 d3b1 d8d1+ c1d1 
5 -60 14 84645 f2f4 e7c5 f1f2 f8g7 c1d1 
5 -55 34 188628 d3d2 e7d6 g2g3 a8d8 c3d5 
5 -55 36 198217 d3d2 e7d6 g2g3 a8d8 c3d5 
6 -196 43 234447 d3d2 f5f4 c3e2 c7b8 e3c4 g8g2+ 
6 -48 48 263396 c3d5 c7d6 c1c7 b7d5 e3d5 e6d5 
6 -48 74 393380 c3d5 c7d6 c1c7 b7d5 e3d5 e6d5 
7 -60 88 477101 g2g3 c7d6 d3c4 f8g7 f1d1 d6e5 d1d7 
7 -56 116 603413 d3e2 e7d6 e2h5 g8g7 c3b5 d6h2+ h5h2 
7 -55 143 701444 c3a4 c7d8 d3e2 f8g7 a4c5 e7c5 c1c5 
7 -55 177 847414 c3a4 c7d8 d3e2 f8g7 a4c5 e7c5 c1c5 
8 -16 190 928375 c3a4 c7d8 d3d8+ a8d8 f2f4 d8d2 c1c7 b7g2 
8 -16 200 987934 c3a4 c7d8 d3d8+ a8d8 f2f4 d8d2 c1c7 b7g2 
9 -66 253 1208560 c3a4 c7d8 d3c4 b7d5 e3d5 d8d5 c4d5 e6d5 a4c5 
9 -63 309 1541791 f1d1 a8d8 d3f1 d8d1 f1d1 e7d6 d1h5 g8g7 c1d1 
9 -51 380 1952172 g2g3 c7d6 d3e2 b7c6 e3c4 d6d4 c1d1 d4c5 f1e1 
9 -51 469 2444824 g2g3 c7d6 d3e2 b7c6 e3c4 d6d4 c1d1 d4c5 f1e1 
10 -51 1486 8063110 g2g3 c7f4 e3c4 f4c7 c4e3 c7f4 e3c4 f4c7 c4e3 
10 -51 1512 8207175 g2g3 c7f4 e3c4 f4c7 c4e3 c7f4 e3c4 f4c7 c4e3 
11 -60 1672 9108656 g2g3 c7f4 e3c4 a8d8 d3e2 e7f6 c4e3 f6e5 f1e1 h7h6 a2a3 
11 -53 2455 13245257 c3e2 c7b6 f1e1 e7c5 e2f4 a8d8 d3c3 c5d4 c3a3+ d8d6 c1d1 
11 -53 2939 15968079 c3e2 c7b6 f1e1 e7c5 e2f4 a8d8 d3c3 c5d4 c3a3+ d8d6 c1d1

Code: Select all

LMR OFF
1 -868 0 10 d3a6 
1 -210 0 51 e3f5 
1 -84 0 73 c3d5 
1 -74 0 146 c3e4 
1 -49 0 171 f2f4 
1 -49 0 253 f2f4 
2 6 0 539 f2f4 f8g7 
2 22 0 1188 c3d5 c7d8 
2 22 0 2394 c3d5 c7d8 
3 -214 0 4697 c3d5 c7d6 e3f5 
3 -81 1 7011 f2f4 b7g2 e3g2 
3 -54 1 11268 h2h3 e7c5 d3c4 
3 -54 2 16319 h2h3 e7c5 d3c4 
4 1 4 26635 h2h3 e7d6 g2g3 f8g7 
4 7 5 35386 f1d1 f8g7 c3d5 c7d6 
4 7 10 57219 f1d1 f8g7 c3d5 c7d6 
5 -59 19 115081 f1d1 a8d8 d3e2 d8d1+ c1d1 
5 -56 47 289041 d3e2 e7d6 e2h5 g8g7 f1d1 
5 -56 51 314736 d3e2 e7d6 e2h5 g8g7 f1d1 
6 -63 79 473848 d3e2 f5f4 e3g4 c7c6 e2f3 c6f3 
6 -45 129 748759 h2h3 f5f4 c3e2 c7e5 e3g4 e5b2 
6 -35 161 919638 f1d1 c7b8 h2h3 e7c5 a2a3 c5e3 
6 -6 204 1152748 c3a4 b7c6 d3c4 f5f4 c4c6 c7c6 
6 0 240 1356259 c3e2 c7b6 g2g3 f8g7 e3c4 b6c5 

Stockfish(only psqt) finds it at depth 11 (using 91ms of time; looking at 131840 nodes so more than 100 times less than weini depth 11 !!! ...) and the search is stable after that

Code: Select all

info depth 1 seldepth 1 multipv 1 score cp 3 nodes 59 nps 29500 tbhits 0 time 2 pv c3d5
info depth 2 seldepth 2 multipv 1 score cp 623 nodes 111 nps 55500 tbhits 0 time 2 pv c3d5 b7d5 c1c7 d5a2
info depth 3 seldepth 3 multipv 1 score cp 623 nodes 165 nps 82500 tbhits 0 time 2 pv c3d5 b7d5 c1c7
info depth 4 seldepth 4 multipv 1 score cp 623 nodes 232 nps 116000 tbhits 0 time 2 pv c3d5 b7d5 c1c7 d5a2
info depth 5 seldepth 5 multipv 1 score cp 647 nodes 373 nps 186500 tbhits 0 time 2 pv c3d5 b7d5 c1c7 d5a2 f1c1
info depth 6 seldepth 6 multipv 1 score cp 730 nodes 882 nps 294000 tbhits 0 time 3 pv c3d5 b7d5 c1c7 d5f3 f1b1 a6a5
info depth 7 seldepth 7 multipv 1 score cp 345 nodes 1428 nps 476000 tbhits 0 time 3 pv c3d5 c7c1 f1c1 b7d5 e3d5 e6d5 d3d5
info depth 8 seldepth 10 multipv 1 score cp -5 nodes 18676 nps 1245066 tbhits 0 time 15 pv f1d1 c7f4 g2g3 f4b4 d1d2 f8g7 d3d7 g8d8
info depth 9 seldepth 11 multipv 1 score cp -37 nodes 33807 nps 1300269 tbhits 0 time 26 pv d3d4 c7d6 c3e2 d6d8 d4c3 e7f6 c3c7 d8e7 c7e7 f6e7
info depth 10 seldepth 14 multipv 1 score cp -115 nodes 109254 nps 1418883 tbhits 0 time 77 pv c3a4 c7f4 a4b6 f4f3 b6a8 g8g2 g1h1 g2f2 h1g1 f2f1 c1f1 f3h1 g1f2 h1h2 f2e1 b7a8
info depth 11 seldepth 18 multipv 1 score cp -60 nodes 131840 nps 1448791 tbhits 0 time 91 pv c3e2 c7b6 d3c3 a8c8 c3e5 e7c5 h2h3 b6c7 e5c7 c8c7 a2a3

It seems that Stockfish use very aggressive pruning before depth 7 ! isn't that very risky to prune that much initially ? I understand this is rewarding a lot as the tree will be much much smaller.

I think I need to implement some stuff about bad capture ordering. Weini is looking at all captures first just after PV/TT move(s) ordered by MVV-LVA because my SEE is always too slow ... (I'm losing more or less as much time doing SEE that I gain from the better ordering that is gives).

Code: Select all

depth	      Weini	Stockfish	ratio
1           253            59     4,2881355932
2          2394           111    21,5675675676
3         16319           165    98,903030303
4         37623           232   162,1681034483
5        198217           373   531,4128686327
6        393380           882   446,0090702948
7        847414          1428   593,4271708683
8        987934         18676    52,8985864211
9       2444824         33807    72,3170940929
10      9108656        109254    83,3713731305
11     15968079        131840   121,1171040655