Obligatory scaling

[Lyudmil Tsvetkov]

Well, going back from theoretic to very practical scaling, I think one essential, absolutely necessary thing to do is to scale down single rook endgames.

Reaons are simple: single rook endgames represent some 8% of all endgames, while the second most frequent endgame after single rook endgames come at only some 1-2% frequency. So that, if you do not scale single rook endgames, but scale something else instead, you are simply wasting your time.

Almost all single rook endgames are of course very drawish. How I think they could be scaled down reasonably?

1. scale down by 15-20% all single rook endgames, independently of other features (well, why not, even if you have big advantage, the game might still end in a draw, if you have small advantage, the game will certainly end in a draw, so it is better to go into any other endgame excluding opposite bishop endings)

2. scale down by 70% all single rook endgames with just 4 vs 3 pawns on one and the same side (either queen or king, usually king). This is very reasonable to do, as most such games end in a draw, no matter what the engines show

3. scale down by 90% all single rook endgames with just 3 vs 2 pawns on one wing of the board (and of course, that also means that you should scale all such endgames with equal number of pawns on one wing of the board, 3 vs 3, or all endgames with pawns just on one wing, f,g,h or a,b,c files, when the pawns are equal or one side has just a pawn more, 2 vs 2, 2 vs 1, etc.)
90% decrease of score is not big at all, as such endings are simply drawn without exception

[d]4r1k1/5pp1/7p/8/3R4/4P2P/5PP1/6K1 w - - 0 58
most likely a draw, but engines do not think like that

[d]4r1k1/6p1/7p/8/3R4/7P/5PP1/6K1 w - - 0 58
draw guaranteed, but engines do not think like that

[d]4r1k1/6p1/8/8/3R4/7P/5P2/6K1 w - - 0 58
even simpler draw, but some engines might want to play on

Are there programmers/engines among you that do not scale single rook endgames?
What do you think about single rook endgames being 10 times as frequent as the next most frequent ending?

[tpetzke]

Can you backup your feeling with statistics data, I mean not that rook endgames are frequent. I'm with you about that, but that 90% of 3 vs 2 pawns end in a draw when the pawns are on one side. Seems a bit drastic to me.

The engine will probably encounter during search a lot of positions with double and even triple pawns that are not drawish at all and such a simple scaling will probably hurt quite a lot.

Thomas...

[Lyudmil Tsvetkov]

I do not know what game to post as an example. Whatever single rook endgames you look at, they seem to be drawn, unless the stronger side has very substantial advantage.

Below a random sample game.

[pgn][PlyCount "190"]
[EventType "simul"]
[Source "Doe"]
[MLNrOfMoves "95"]
[MLFlags "010100"]
[Event "201403Stockfish5_4+0_240games"]
[Date "2014.06.01"]
[Round "23.1"]
[White "Stockfish 5 64 SSE4.2x6"]
[Black "Houdini 4 x64_st_X6_CT0"]
[Result "1/2-1/2"]
[EventDate "2014.05.31"]
[ECO "B07"]
[TimeControl "240"]
[Annotator "0.35;0.16"]


{Intel(R) Core(TM) i7 CPU X 980 @ 3.33GHz 3337 MHz W=24.6 plies; 9. 968kN/s; Perfect2014c.ctg B=20.4 plies; 12.642kN/s; 3.386 TBAs; Perfect2014c. ctg}
1. e4 {[%eval 0,0] [%emt 0:00:00]} 1... g6 {[%eval 0,0] [%emt 0:00:00]} 2. d4
{[%eval 0,0] [%emt 0:00:00]} 2... Bg7 {[%eval 0,0] [%emt 0:00:00]} 3. Nc3
{[%eval 0, 0] [%emt 0:00:00]} 3... d6 {[%eval 0,0] [%emt 0:00:00]} 4. Be3
{[%eval 0,0] [%emt 0: 00:00]} 4... Nf6 {[%eval 0,0] [%emt 0:00:00]} 5. Qd2
{[%eval 0,0] [%emt 0:00:00]} 5... c6 {[%eval 0,0] [%emt 0:00:00]} 6. Bh6
{[%eval 35,23] [%emt 0:00:15]} 6... O-O {[%eval 16,21] [%emt 0:00:07]} 7. Bxg7
{[%eval 20,23] [%emt 0:00:05] (Cf3)} 7... Kxg7 {[%eval 16,20] [%emt 0:00:01]} 8.
Be2 {[%eval 27,23] [%emt 0:00:06] (Cf3)} 8... c5
{[%eval 2,20] [%emt 0:00:05] (Db6)} 9. dxc5 {[%eval 36,23] [%emt 0:00:11] (d5)}
9... dxc5 {[%eval 12,19] [%emt 0:00:03]} 10. Qg5 {[%eval 37,24] [%emt 0:00:06]}
10... Qb6 {[%eval 18,21] [%emt 0:00:03] (Da5)} 11. O-O-O
{[%eval 63,24] [%emt 0:00:03]} 11... Nc6 {[%eval 10,21] [%emt 0:00:04] (h6)} 12.
e5 {[%eval 66,21] [%emt 0:00:03]} 12... Nd7 {[%eval 20,20] [%emt 0:00:03] (Ce8)}
13. f4 {[%eval 66,22] [%emt 0:00:04]} 13... f6 {[%eval 16,19] [%emt 0:00:03]}
14. exf6+ {[%eval 59,23] [%emt 0:00:03]} 14... Nxf6
{[%eval 35,20] [%emt 0:00:13]} 15. Nf3 {[%eval 51,23] [%emt 0:00:05]} 15... Bg4
{[%eval 18,20] [%emt 0:00:05] (Af5)} 16. Rhe1
{[%eval 74,22] [%emt 0:00:03] (Ce5)} 16... Rad8 {[%eval 36,19] [%emt 0:00:07]}
17. h3 {[%eval 67,22] [%emt 0:00:04] (Txd8)} 17... Bf5
{[%eval 16,19] [%emt 0:00:03]} 18. Ne5 {[%eval 52,23] [%emt 0:00: 09] (Txd8)}
18... Nd4 {[%eval 32,18] [%emt 0:00:08]} 19. Bd3 {[%eval 57,25] [%emt 0: 00:04]}
19... Bxd3 {[%eval 18,19] [%emt 0:00:04]} 20. Rxd3
{[%eval 44,24] [%emt 0:00: 06]} 20... Qc7 {[%eval 18,19] [%emt 0:00:02]} 21.
Rde3 {[%eval 41,25] [%emt 0:00:09] (Cf3)} 21... Nf5
{[%eval 16,20] [%emt 0:00:06] (e6)} 22. R3e2 {[%eval 41,23] [%emt 0: 00:06]}
22... Nd4 {[%eval 25,20] [%emt 0:00:03]} 23. Rf2 {[%eval 43,23] [%emt 0:00: 03]}
23... e6 {[%eval 20,20] [%emt 0:00:02]} 24. g4 {[%eval 55,23] [%emt 0:00:02]}
24... Kh8 {[%eval 40,20] [%emt 0:00:25]} 25. Nb1
{[%eval 55,23] [%emt 0:00:05] (h4)} 25... b5
{[%eval 48,18] [%emt 0:00:06] (Tde8)} 26. Qh6
{[%eval 59,24] [%emt 0:00:04] (Cd2)} 26... Kg8 {[%eval 43,16] [%emt 0:00:02]}
27. c3 {[%eval 60,26] [%emt 0:00:03] (Cd2)} 27... Nc6
{[%eval 47,16] [%emt 0:00:01]} 28. Nxc6 {[%eval 68,27] [%emt 0:00: 02]} 28...
Qxc6 {[%eval 47,15] [%emt 0:00:00]} 29. f5 {[%eval 53,26] [%emt 0:00:04]} 29...
exf5 {[%eval 38,18] [%emt 0:00:03]} 30. gxf5 {[%eval 62,28] [%emt 0:00:02]}
30... Nd7 {[%eval 48,20] [%emt 0:00:02]} 31. Rfe2 {[%eval 67,27] [%emt 0:00:04]}
31... Qf6 {[%eval 48,18] [%emt 0:00:00]} 32. fxg6 {[%eval 75,27] [%emt 0:00:02]}
32... Qxg6 {[%eval 36,18] [%emt 0:00:02]} 33. Qxg6+
{[%eval 75,25] [%emt 0:00:02] (De3)} 33... hxg6 {[%eval 33,17] [%emt 0:00:00]}
34. Nd2 {[%eval 67,25] [%emt 0:00:03]} 34... b4 {[%eval 39,21] [%emt 0:00:04]}
35. Ne4 {[%eval 63,26] [%emt 0:00:07]} 35... bxc3 {[%eval 35,20] [%emt 0:00:01]}
36. Nxc3 {[%eval 69,26] [%emt 0:00:02]} 36... c4 {[%eval 37,21] [%emt 0:00:01]}
37. Re6 {[%eval 78,26] [%emt 0:00:02]} 37... Nc5
{[%eval 31,20] [%emt 0:00:02] (Tf6)} 38. Rxg6+ {[%eval 100,23] [%emt 0:00:03]}
38... Kh7 {[%eval 31,22] [%emt 0:00:02]} 39. Reg1
{[%eval 100,26] [%emt 0:00:03]} 39... Nd3+ {[%eval 33,20] [%emt 0:00:01]} 40.
Kb1 {[%eval 91,27] [%emt 0:00:07]} 40... Rf1+ {[%eval 29,21] [%emt 0:00:02]} 41.
Rxf1 {[%eval 91,27] [%emt 0:00:01]} 41... Kxg6 {[%eval 27,20] [%emt 0:00:00]}
42. Nd1 {[%eval 91,26] [%emt 0:00:01]} 42... Re8
{[%eval 27,21] [%emt 0:00:03] (Tb8)} 43. Kc2 {[%eval 88,22] [%emt 0:00:02]}
43... Nb4+ {[%eval 28,19] [%emt 0:00:02]} 44. Kc3 {[%eval 87,24] [%emt 0:00:01]}
44... Nxa2+ {[%eval 28,18] [%emt 0:00:00]} 45. Kxc4
{[%eval 87,24] [%emt 0:00:02]} 45... Nc1 {[%eval 26,19] [%emt 0:00:01] (Tc8+)}
46. Kc3 {[%eval 79,22] [%emt 0:00:04] (Tf3)} 46... Kg5
{[%eval 21,19] [%emt 0:00:02] (Tc8+)} 47. Rf7
{[%eval 77,19] [%emt 0:00:02] (Tf3)} 47... a5 {[%eval 19,19] [%emt 0:00:04]} 48.
Kd2 {[%eval 76,23] [%emt 0:00:02] (Td7)} 48... Nb3+
{[%eval 17,20] [%emt 0:00:01]} 49. Kd3 {[%eval 70,23] [%emt 0:00:02]} 49...
Rd8+ {[%eval 17,21] [%emt 0:00:04] (Cc1+)} 50. Kc3
{[%eval 73, 24] [%emt 0:00:02]} 50... Nd4 {[%eval 23,19] [%emt 0:00:01] (Cc1)}
51. Rg7+ {[%eval 84,20] [%emt 0:00:02]} 51... Kf6
{[%eval 19,20] [%emt 0:00:01] (Rf5)} 52. Rg4
{[%eval 73,22] [%emt 0:00:03] (Tg2)} 52... Ne2+ {[%eval 14,19] [%emt 0:00:01]}
53. Kc2 {[%eval 66,25] [%emt 0:00:04]} 53... Nd4+ {[%eval 12,21] [%emt 0:00:01]}
54. Kb1 {[%eval 66,25] [%emt 0:00:01]} 54... Nc6 {[%eval 14,20] [%emt 0:00:01]}
55. Nf2 {[%eval 63,25] [%emt 0:00:01]} 55... Kf5
{[%eval 14,20] [%emt 0:00:01] (Cb4)} 56. Kc2 {[%eval 78,21] [%emt 0:00:02]}
56... Nb4+ {[%eval 19,20] [%emt 0:00:01]} 57. Kc3
{[%eval 69,22] [%emt 0:00:01] (Rb3)} 57... Nd5+
{[%eval 16,19] [%emt 0:00:01] (Te8)} 58. Kb3 {[%eval 72,22] [%emt 0:00:01]}
58... Rb8+ {[%eval 10,20] [%emt 0:00:01] (Cf4)} 59. Ka3
{[%eval 72,24] [%emt 0:00:02]} 59... Ne3 {[%eval 10,23] [%emt 0:00:02] (Cf4)}
60. Ra4 {[%eval 58,24] [%emt 0:00:01]} 60... Nc2+
{[%eval 10,22] [%emt 0:00: 01]} 61. Ka2 {[%eval 58,1] [%emt 0:00:00]} 61... Nb4+
{[%eval 11,23] [%emt 0:00:02]} 62. Kb1 {[%eval 58,25] [%emt 0:00:01]} 62... Rb5
{[%eval 11,23] [%emt 0:00:01]} 63. Ra3 {[%eval 60,25] [%emt 0:00:01]} 63... Rd5
{[%eval 11,22] [%emt 0:00:01]} 64. Rf3+ {[%eval 60,25] [%emt 0:00:01]} 64... Ke5
{[%eval 9,22] [%emt 0:00:01] (Rg5)} 65. Kc1 {[%eval 63,24] [%emt 0:00:02]} 65...
Rd8 {[%eval 12,21] [%emt 0:00:01] (Td4)} 66. Ra3
{[%eval 65,24] [%emt 0:00:01] (h4)} 66... Rd5
{[%eval 12,20] [%emt 0:00:02] (Tc8+)} 67. Nd1 {[%eval 62,21] [%emt 0:00:01]}
67... Kf4 {[%eval 12,21] [%emt 0:00:02] (Cd3+)} 68. Nc3
{[%eval 86,23] [%emt 0:00:01]} 68... Rf5 {[%eval 10,22] [%emt 0:00:01] (Td8)}
69. Ne2+ {[%eval 67,20] [%emt 0:00:01]} 69... Kg5
{[%eval 10,20] [%emt 0:00: 00]} 70. Nd4 {[%eval 67,25] [%emt 0:00:01] (Rd2)}
70... Rd5 {[%eval 9,20] [%emt 0:00: 01] (Tf1+)} 71. Nf3+
{[%eval 69,21] [%emt 0:00:01]} 71... Kf5 {[%eval 10,21] [%emt 0: 00:01] (Rf4)}
72. Ne1 {[%eval 65,23] [%emt 0:00:01]} 72... Ke4 {[%eval 10,20] [%emt 0:00:01]}
73. Rg3 {[%eval 58,23] [%emt 0:00:01]} 73... Rf5 {[%eval 10,21] [%emt 0:00: 01]}
74. Rg4+ {[%eval 58,22] [%emt 0:00:01]} 74... Ke5
{[%eval 12,20] [%emt 0:00:04] (Rd5)} 75. Kd1 {[%eval 53,21] [%emt 0:00:01] (h4)}
75... Rf1 {[%eval 12,20] [%emt 0: 00:01]} 76. Ke2
{[%eval 53,22] [%emt 0:00:01] (Th4)} 76... Rh1 {[%eval 16,18] [%emt 0:00:01]}
77. Rg3 {[%eval 48,22] [%emt 0:00:01]} 77... Kf4
{[%eval 18,19] [%emt 0:00: 01] (Th2+)} 78. Rf3+ {[%eval 54,24] [%emt 0:00:01]}
78... Ke4 {[%eval 18,20] [%emt 0: 00:01]} 79. Re3+
{[%eval 54,24] [%emt 0:00:00]} 79... Kf4 {[%eval 14,21] [%emt 0:00: 01]} 80.
Ng2+ {[%eval 54,23] [%emt 0:00:00]} 80... Kf5 {[%eval 11,21] [%emt 0:00:03]} 81.
Rg3 {[%eval 56,23] [%emt 0:00:00]} 81... Ke6
{[%eval 14,20] [%emt 0:00:01] (Re5)} 82. Nf4+
{[%eval 58,21] [%emt 0:00:00] (Ta3)} 82... Ke5
{[%eval 12,20] [%emt 0:00:01] (Rf5)} 83. Nd3+ {[%eval 58,25] [%emt 0:00:01]}
83... Nxd3 {[%eval 14,21] [%emt 0:00: 01]} 84. Kxd3
{[%eval 58,28] [%emt 0:00:00]} 84... a4 {[%eval 13,22] [%emt 0:00:01]} 85. Kc2
{[%eval 58,29] [%emt 0:00:00]} 85... Kd5 {[%eval 16,22] [%emt 0:00:01] (Rf5)}
86. Re3 {[%eval 58,32] [%emt 0:00:00] (Rc3)} 86... Kd4
{[%eval 13,24] [%emt 0:00:01]} 87. Rf3 {[%eval 58,32] [%emt 0:00:00] (Tc3)}
87... Kc4 {[%eval 13,24] [%emt 0:00:01] (Re4)} 88. Rc3+
{[%eval 17,26] [%emt 0:00:01] (Tf4+)} 88... Kd4
{[%eval 14,24] [%emt 0:00:01] (Rd5)} 89. Ra3
{[%eval 17,33] [%emt 0:00:00] (Td3+)} 89... Kd5
{[%eval 10, 24] [%emt 0:00:01] (Rc5)} 90. Rg3 {[%eval 17,30] [%emt 0:00:01]}
90... Kc5 {[%eval 12,23] [%emt 0:00:01] (Rd6)} 91. Rd3
{[%eval 17,28] [%emt 0:00:00] (Tg5+)} 91... Kb5
{[%eval 9,24] [%emt 0:00:00] (Rc4)} 92. Rc3 {[%eval 17,33] [%emt 0:00:01] (Te3)}
92... Kb4 {[%eval 9,25] [%emt 0:00:00]} 93. Re3 {[%eval 16,33] [%emt 0:00:00]}
93... Kc5 {[%eval 9,25] [%emt 0:00:01] (Rb5)} 94. Re5+
{[%eval 16,32] [%emt 0:00:00]} 94... Kd4 {[%eval 9,26] [%emt 0:00:00]} 95. Ra5
{[%eval 10,31] [%emt 0:00:00]} 95... Rh2+ {[%eval 9,25] [%emt 0:00:01]} 1/2-1/2
[/pgn]

[d]8/8/8/4k3/p7/3K2RP/1P6/7r w - - 0 85
SF thinks here it has some 60cps advantage, but this is a simple draw.
One pawn material advantage will usually not be sufficient for a win, if the weaker side is active, especially if its rook attacks enemy pawns instead of staying into the rear.

So that you can safely scale by a smaller percentage also all single rook endgames, regardless of the pawn span.

[Vinvin]

...
4r1k1/5pp1/7p/8/3R4/4P2P/5PP1/6K1 w - - 0 58
most likely a draw, but engines do not think like that

4r1k1/6p1/7p/8/3R4/7P/5PP1/6K1 w - - 0 58
draw guaranteed, but engines do not think like that

4r1k1/6p1/8/8/3R4/7P/5P2/6K1 w - - 0 58
even simpler draw, but some engines might want to play on

Are there programmers/engines among you that do not scale single rook endgames?
What do you think about single rook endgames being 10 times as frequent as the next most frequent ending?

During the world blitz championship, the commentator (a GM) gives a positions like this as a theoretical draw too :
[d]8/1r2kpp1/7p/2R5/5PK1/5P2/8/8 w - - 0 1

[Lyudmil Tsvetkov]

Can you backup your feeling with statistics data, I mean not that rook endgames are frequent. I'm with you about that, but that 90% of 3 vs 2 pawns end in a draw when the pawns are on one side. Seems a bit drastic to me.

The engine will probably encounter during search a lot of positions with double and even triple pawns that are not drawish at all and such a simple scaling will probably hurt quite a lot.

Thomas...

Well, I would say that not 90%, but 100 of single rook endgames and 3 vs 2 pawns on just one wing end in a draw. There are simply no winning chances even if there is a double pawn for the weaker side. But I am not very good at statistics and working with applications, perhaps someone else can provide some data for me. Besides, doubled pawns really happen very infrequently with such endings.

The important thing is that you specify the scaling conditions correctly:

- pawn span less or equal to 3
- only one rook each side
- number of pawns equal or just one pawn more for one side

It would be interesting for an engine to encounter during search a triple pawn for the weaker side with 3 vs 2.

[Lyudmil Tsvetkov]

...
4r1k1/5pp1/7p/8/3R4/4P2P/5PP1/6K1 w - - 0 58
most likely a draw, but engines do not think like that

4r1k1/6p1/7p/8/3R4/7P/5PP1/6K1 w - - 0 58
draw guaranteed, but engines do not think like that

4r1k1/6p1/8/8/3R4/7P/5P2/6K1 w - - 0 58
even simpler draw, but some engines might want to play on

Are there programmers/engines among you that do not scale single rook endgames?
What do you think about single rook endgames being 10 times as frequent as the next most frequent ending?

During the world blitz championship, the commentator (a GM) gives a positions like this as a theoretical draw too :
[d]8/1r2kpp1/7p/2R5/5PK1/5P2/8/8 w - - 0 1

Precisely, this is a theoretical draw.

But let SF calculate on it, I bet it sees big black advantage.
Tablebases are still far away, and such a score could sufficiently mislead the engine.

[Uri Blass]

Can you backup your feeling with statistics data, I mean not that rook endgames are frequent. I'm with you about that, but that 90% of 3 vs 2 pawns end in a draw when the pawns are on one side. Seems a bit drastic to me.

The engine will probably encounter during search a lot of positions with double and even triple pawns that are not drawish at all and such a simple scaling will probably hurt quite a lot.

Thomas...

Well, I would say that not 90%, but 100 of single rook endgames and 3 vs 2 pawns on just one wing end in a draw.

100% is obviously wrong

[D]5rk1/4K1pp/8/5PPP/8/8/8/1R6 w - - 12 1

white wins by 1.g6

[Lyudmil Tsvetkov]

Can you backup your feeling with statistics data, I mean not that rook endgames are frequent. I'm with you about that, but that 90% of 3 vs 2 pawns end in a draw when the pawns are on one side. Seems a bit drastic to me.

The engine will probably encounter during search a lot of positions with double and even triple pawns that are not drawish at all and such a simple scaling will probably hurt quite a lot.

Thomas...

Well, I would say that not 90%, but 100 of single rook endgames and 3 vs 2 pawns on just one wing end in a draw.

100% is obviously wrong

[D]5rk1/4K1pp/8/5PPP/8/8/8/1R6 w - - 12 1

white wins by 1.g6

This is a position with very big white additional advantage:

- all white pawns on the 5th rank, all black on the 2nd
- much better white king psqt
- very passive black rook
- side to move white

plus some very peculiar specific features also favouring white, like a mating net after g6

[Harald]

Your rule just counts the black and white pawns on one side.
I assume 4 vs. 2 or 1 or 0 pawns are not part of this rule, right?
(The same for 3 vs. 1 or 0 and 2 vs. 0.)
What if there are 5 pawns for one side?
What if there are double or triple pawns?
What if some of the pawns have already passed each other somehow?
What if white pawns are on file e (and f) and black pawns on h (and g)?

In this and other rules I am not sure how detailed you have looked
at _all_ possibilities of the distribution of pawns and pieces on the board.
This is interesting from a programmer's point of view. We need every
trick and bitmask and shortcut for an implementation of the rules
including absurd combinations that would only happen on a board
directly from a setup and not from a game.

Harald

[tpetzke]

[D]8/5PP1/4K3/4R2p/6kp/7p/7r/8 w - - 0 1

I think I can win this as white

Thomas...