Brand-new Stockfish tablebase test

[Jouni]

From SF wiki:
--------------------------------------------------
Results of Stockfish vs StockfishSyzygy (10+0.1, 1t, 16MB, UHO_XXL_+0.90_+1.19.pgn):
Elo: -1.26 +/- 1.46, nElo: -2.64 +/- 3.05
LOS: 4.46 %, DrawRatio: 57.40 %, PairsRatio: 0.96
Games: 50000, Wins: 13671, Losses: 13853, Draws: 22476, Points: 24909.0 (49.82 %)
Ptnml(0-2): [130, 5293, 14349, 5085, 143], WL/DD Ratio: 1.37
--------------------------------------------------

Stockfish 17.1 with 6men TB on very fast M2 SSD. With longer time control probably less gain.

[Joerg Oster]

From SF wiki:

--------------------------------------------------
Results of Stockfish vs StockfishSyzygy (10+0.1, 1t, 16MB, UHO_XXL_+0.90_+1.19.pgn):
Elo: -1.26 +/- 1.46, nElo: -2.64 +/- 3.05
LOS: 4.46 %, DrawRatio: 57.40 %, PairsRatio: 0.96
Games: 50000, Wins: 13671, Losses: 13853, Draws: 22476, Points: 24909.0 (49.82 %)
Ptnml(0-2): [130, 5293, 14349, 5085, 143], WL/DD Ratio: 1.37
--------------------------------------------------

Stockfish 17.1 with 6men TB on very fast M2 SSD. With longer time control probably less gain.

I wonder if the mostly already decided openings play a role here ...
Also note, this is foremost testing the WDL part of the bases. How much are the DTZ files?
Especially when it comes to those rather difficult rook or queen endgames?!

Edit and P.S.: Of course, modern engines simply became so strong that endgame bases make no measurable difference anymore.
However, it always has been difficult to measure endgame knowledge ...

[Jouni]

In my own test playing directly endings common thing is, that TBs have negative effect to play until may be 1000 games. Then they are stored to RAM to speed up access.

[syzygy]

In my own test playing directly endings common thing is, that TBs have negative effect to play until may be 1000 games. Then they are stored to RAM to speed up access.

I suspect the speed "penalty" has become very small now on modern computers with modern SSDs (and sufficient RAM), and with Stockfish now having a much heavier (NNUE) evaluation than in 2013.

[Ajedrecista]

Hello Jörg:

From SF wiki:

--------------------------------------------------
Results of Stockfish vs StockfishSyzygy (10+0.1, 1t, 16MB, UHO_XXL_+0.90_+1.19.pgn):
Elo: -1.26 +/- 1.46, nElo: -2.64 +/- 3.05
LOS: 4.46 %, DrawRatio: 57.40 %, PairsRatio: 0.96
Games: 50000, Wins: 13671, Losses: 13853, Draws: 22476, Points: 24909.0 (49.82 %)
Ptnml(0-2): [130, 5293, 14349, 5085, 143], WL/DD Ratio: 1.37
--------------------------------------------------

Stockfish 17.1 with 6men TB on very fast M2 SSD. With longer time control probably less gain.

I wonder if the mostly already decided openings play a role here ...
[...]

I find amazing that 'mostly already decided openings' still led to circa 44.95% of draws. Please note that the value of DrawRatio of 57.40% refers to 1 at pentanomial(0-2): 14349/(130 + 5293 + 14349 + 5085 + 143) = 14349/25000 = 57.396% ~ 57.40%. I have just realized about it. Before that, I thought that DrawRatio in these stats was draws/games.

PairsRatio comes from pentanomial(0-2) again: (143 + 5085)/(5293 + 130) ~ 0.9640 ~ 0.96. In other words, the ratio of won pairs and lost pairs. Sign(PairsRatio - 1) = Sign(Elo) = Sign(normalized Elo) = Sign(LOS - 50%), of course.

WL/DD is the ratio of win-lose pairs and all-draw pairs in 1 at pentanomial(0-2). Once the number of draws in 1 at pentanomial(0-2) is calculated: 22476 - 5085 - 5293 = 12098, then there are 12098/2 = 6049 all-draw pairs and 14349 - 6049 = 8300 win-lose pairs, thus WL/DD Ratio = 8300/6049 ~ 1.3721 ~ 1.37. There were 6049 out of 25000 pairs (almost 24.2%) with two draws despite the almost one-sided openings!

Regards from Spain.

Ajedrecista.

[jp]

...
I find amazing that 'mostly already decided openings' still led to circa 44.95% of draws. ...

Um, how do you see that (from the quoted results)?

[Ajedrecista]

Hello:

...
I find amazing that 'mostly already decided openings' still led to circa 44.95% of draws. ...

Um, how do you see that (from the quoted results)?

Draws/Games = 22476/50000 ~ 44.95%.

'Mostly already decided openings' was a quote by Jörg. I guess that the name of the opening book tells that the exit positions of the book are evaluated between +0.90 and +1.19 by SF at a certain depth. I remember that a eval of +1 at move 32 (much later than the exit positions of the book) means 50% of chances of a win, unless recent changes that I am not aware.

Regards from Spain.

Ajedrecista.